10 May 2014
Missed what’s been happening in the data management community? Check out our bi-weekly update:
Agile Amazon analytics Apple Big Data books Business Intelligence cloud Cloud computing Collaboration consultants CRM data governance data modelling data privacy data quality data standards data visualization data warehousing digital disruption ERP ETL executive issues facebook Google Hadoop IBM Information Development Information Governance information theory Information Value Internet of Things metadata management Microsoft Netflix omCollab open data Open Source people Privacy security social media SQL Twitter unstructured data
Archive for the ‘Information Development’ Category
10 May 2014
Missed what’s been happening in the data management community? Check out our bi-weekly update:
01 May 2014
Today, many organizations are facing increased scrutiny and a higher level of overall performance expectation from internal and external stakeholders. Both business and public sector leaders must provide greater external and internal transparancy to their activities, ensure accounting data faces up to compliance challenges, and extract the return and competitive advantage out of their customer, operational and performance information: Managers, investors and regulators have a new perspective on performance and compliance, which may include:
Senior executives across industry agree that achieving growth and performance objectives requires improved analytics, defining and implementing a process to track, monitor and measure innovation and performance, and making available more timely, accurate information for decision-making. MIKE2.0′s open source Enterprise Performance Management offering can help organizations achieve these critical objectives.
26 Apr 2014
22 Apr 2014
On the recent Stuff to Blow Your mind podcast episode Outsourcing Memory, hosts Julie Douglas and Robert Lamb discussed how, from remembering phone numbers to relying on spellcheckers, we’re allocating our cognitive processes to the cloud.
“Have you ever tried to recall an actual phone number stored in your cellphone, say of a close friend or relative, and been unable to do so?” Douglas asked. She remarked how that question would have been ridiculous ten years ago, but nowadays most of us would have to admit that the answer is yes. Remembering phone numbers is just one example of how we are outsourcing our memory. Another is spelling. “Sometimes I find myself intentionally misspelling a word to make sure the application I am using is running a spellchecker,” Lamb remarked. Once confirmed, he writes without worrying about misspellings since the spellchecker will catch them. I have to admit that I do the same thing. In fact, while writing this paragraph I misspelled several words without worry since they were automatically caught by those all-too-familiar red-dotted underlines. (Don’t forget, however, that spellcheckers don’t check for contextual accuracy.)
Transactive Memory and Collaborative Remembering
Douglas referenced the psychological concept of transactive memory, where groups collectively store and retrieve knowledge. This provides members with more and better knowledge than any individual could build on their own. Lamb referenced cognitive experimental research on collaborative remembering. This allows a group to recall information that its individual members had forgotten.
The memory management model of what we now call the cloud is transactive memory and collaborative remembering on a massive scale. It has pervaded most aspects of our personal and professional lives. Douglas and Lamb contemplated both its positive and negative aspects. Many of the latter resonated with points I made in my previous post about Automation and the Danger of Lost Knowledge.
Free Your Mind
In a sense, outsourcing our memory to the cloud frees up our minds. It is reminiscent of Albert Einstein remarking that he didn’t need to remember basic mathematical equations since he could just look them up in a book when he needed them. Nowadays he would just look them up on Google or Wikipedia (or MIKE2.0 if, for example, he needed a formula for calculating the economic value of information). Not bothering to remember basic mathematical equations freed up Einstein’s mind for his thought experiments, allowing him to contemplate groundbreaking ideas like the theory of relativity.
Forgetting how to Remember
I can’t help but wonder what our memory will be like ten years from now after we have outsourced even more of it to the cloud. Today, we don’t have to remember phone numbers or how to spell. Ten years from now, we might not have to remember names or how to count.
Wearable technology, like Google Glass or Narrative Clip, will allow us to have an artificial photographic memory. Lifelogging will allow us to record our own digital autobiography. “We have all forgot more than we remember,” Thomas Fuller wrote in the 18th century. If before the end of the 21st century we don’t have to remember anything, perhaps we will start forgetting how to remember.
I guess we will just have to hope that a few trustworthy people remember how to keep the cloud working.
20 Apr 2014
Just how productive are Chief Information Officers or the technology that they manage? With technology portfolios becoming increasingly complex it is harder than ever to measure productivity. Yet boards and investors want to know that the capital they have tied-up in the information technology of the enterprise is achieving the best possible return.
For CIOs, talking about value improves the conversation with executive colleagues. Taking them aside to talk about the success of a project is, even for the most strategic initiatives, usually seen as a tactical discussion. Changing the topic to increasing customer value or staff productivity through a return on technology capital is a much more strategic contribution.
What is the return on an IT system?
There are all sorts of productivity measures that can be applied to individual systems, but they are usually based on the efficiency of existing processes which leads to behaviours which reduce flexibility. The future of business and government depends on speed of response to change, not how efficiently they deal with a static present.
Businesses invest in information systems to have the right information at the right time to support decisions or processes. Information that is used is productive while information that is collected, but poorly applied, is wasted or unproductive.
However, to work out what proportion of information is being used there needs to be a way to quantify it.
How much information is contained in the systems?
There is a formal way to measure the quantity of information. I introduce this extensively in chapter 6 of Information-Driven Business.
The best way to understand “quantity” in terms of information is to count the number of artefacts rather than the number of bits or bytes required to store them. The best accepted approach to describing this quantity is called “information entropy” which, confusingly, uses a “bit” as its unit of measure which is a count of the potential permutations that the system can represent.
A system that holds 65,536 names has just 16 “bits” of unique information (log265536). That might sound strange given that the data storage of 65,536 names might use of the order of 6MB.
To understand why there only 16 bits of unique information in a list of 65,536 names consider whether the business uses the spelling of the names of if there is any additional insight being gained from the data that is stored.
How much of that information is actually used?
Knowing how much information there is in a system opens up the opportunity to find how much information is being productively used. The amount of information being used to drive customer or management choices is perhaps best described as “decision entropy”. The decision entropy is either equal or less than the total information entropy.
An organisation using 100% of their available information is incredibly lean and nimble. They have removed much of the complexity that stymies their competitors (see Value of decommissioning legacy systems).
Of course, no organisation productively uses all of the information that they hold. Knowing that holding unproductive information comes at a cost to the organisation, the CIO can have an engaging conversation with fellow executives about extracting more value from existing systems without changing business processes.
When looking at how business reports are really used, and how many reports lie unread on management desks, there is a lot of low hanging fruit to be picked just by improving the way existing business intelligence is used.
Similarly, customer systems seldom maximise their use of hints based on existing information to guide buyers to close the best available offer. A few digital enhancements at the front line can bring to the surface a vast array of otherwise unused information.
Changing the conversation
Globally, CIOs are finding themselves pushed down a rung in the organisational ladder. This is happening at the very same time that technology is moving from the back office to become a central part of the revenue story through digital disruption.
CIOs are not automatically entitled to be at the executive table. They have to earn the right by contributing to earnings and business outcomes. One of the best discussions for a CIO to focus on is increasing productivity of the capital tied-up in the investments that have already been made in the systems that support staff and customers.
15 Apr 2014
Missed what happened in the MIKE2.0 Community? Here’s a quick recap:
11 Apr 2014
Every organization, regardless of size, understands the importance of good on-boarding procedures for incoming employees and new hires. We have a slew of welcome packets, orientations, procedures and trainings to ensure we’re providing our incoming talent with the right tools to be successful in their new roles. But how often are we taking the same care to off board our departing employees when sensitive company information and intellectual property is at stake?
This topic has really hit home for our organization this year, as we began to develop procedures for operations that haven’t been clearly defined or documented, impacting critical departments such as HR, IT and Finance. Developing SOPs for these “gray areas” of the business has uncovered some interesting gaps that were not previously being closed. In the end, we discovered that while we had very clear instructions for bringing new talent into the company, we had no formal process for those who left and most off boarding activities were being carried out on an ad hoc basis.
Lesson learned: Regardless of company size or resources, taking the right approach to off boarding can save a giant headache when it comes to information security. It should be a preventive measure and not a reactive process.
As a baseline, organizations should give careful thought to the following information access points:
- Phone Directories
- Documents/File Sharing Systems
- CRM/Mailing Lists
- Company Intranets
- Website or FTPs
How well is your team closing the gap with respect to these information access points? Does your HR department communicate off boarding needs to IT, and do employees sign an NDA upon hire? How are you ensuring your intellectual property and other critical enterprise information is being safeguarded from departing talent?
31 Mar 2014
Missed what happened in the MIKE2.0 community this past week? Read on:
30 Mar 2014
While security and privacy issues prevent sensitive data from being shared (e.g., customer data containing personal financial information or patient data containing personal health information), do you have access to data that would be more valuable if you shared it with the rest of your organization—or perhaps the rest of the world?
We are all familiar with the opposite of data sharing within an organization—data silos. Somewhat ironically, many data silos start with data that was designed to be shared with the entire organization (e.g., from an enterprise data warehouse), but was then replicated and customized in order to satisfy the particular needs of a tactical project or strategic initiative. This customized data often becomes obsolesced after the conclusion (or abandonment) of its project or initiative.
Data silos are usually denounced as evil, but the real question is whether the data hoarded within a silo is sharable—is it something usable by the rest of the organization, which may be redundantly storing and maintaining their own private copies of the same data, or are the contents of the data silo something only one business unit uses (or is allowed to access in the case of sensitive data).
Most people decry data silos as the bane of successful enterprise data management—until you expand the scope of data beyond the walls of the organization, where the enterprise’s single version of the truth becomes a cherished data asset (i.e., an organizational super silo) intentionally siloed from the versions of the truth maintained within other organizations, especially competitors.
We need to stop needlessly replicating and customizing data—and start reusing and sharing data.
Historically, replication and customization had two primary causes:
Hoarding data in a proprietary format and viewing “our private knowledge is our power” must be replaced with shared data in an open format and viewing “our shared knowledge empowers us all.”
This is an easier mantra to recite than it is to realize, not only within an organization or industry, but even more so across organizations and industries. However, one of the major paradigm shifts of 21st century data management is making more data publicly available, following open standards (such as MIKE2.0) and using unambiguous definitions so data can be easily understood and reused.
Of course, data privacy still requires sensitive data not be shared without consent, and competitive differentiation still requires intellectual property not be shared outside the organization. But this still leaves a vast amount of data, which if shared, could benefit our increasingly hyper-connected world where most of the boundaries that used to separate us are becoming more virtual every day. Some examples of this were made in the recent blog post shared by Henrik Liliendahl Sørensen about Winning by Sharing Data.
29 Mar 2014
When I was a kid growing up in the UK, Paul Daniels was THE television magician. With a combination of slick high drama illusions, close-up trickery and cheeky end-of-the-pier humour, (plus a touch of glamour courtesy of The Lovely Debbie McGee TM), Paul had millions of viewers captivated on a weekly basis and his cheeky catch-phrases are still recognised to this day.
Of course. part of the fascination of watching a magician perform is to wonder how the trick works. “How the bloody hell did he do that?” my dad would splutter as Paul Daniels performed yet another goofy gag or hair-raising stunt (no mean fear, when you’re as bald as a coot…) But most people don’t REALLY want to know the inner secrets, and ever fewer of us are inspired to spray a riffle-shuffled a pack of cards all over granny’s lunch, stick a coin up their nose or grab the family goldfish from its bowl and hide it in the folds of our nether-garments. (Um, yeah. Let’s not go there…)
Penn and Teller are great of course, because they expose the basic techniques of really old, hackneyed tricks and force more innovation within the magician community. They’re at their most engaging when they actually do something that you don’t get to see the workings of. Illusion maintained, audience entertained.
As data practitioners, I think we can learn a few of these tricks. I often see us getting too hot-and-bothered about differentiating data, master data, reference data, metadata, classification scheme, taxonomy, dimensional vs relational vs data vault modelling etc. These concepts are certainly relevant to our practitioner world, but I don’t necessarily believe they need to be exposed at the business-user level.
For example, I often hear business users talking about “creating the metadata” for an event or transaction, when they’re talking about compiling the picklist of valid descriptive values and mapping these to the contextualising descriptive information for that event (which by my reckoning, really means compiling the reference data!). But I’ve found that business people really aren’t all that bothered about the underlying structure or rigour of the modelling process.
That’s our job.
There will always be exceptions. My good friend and colleague Ben Bor is something a special case and has the talent to combine data management and magic.
But for the rest of us mere mortals, I suggest that we keep the deep discussion of data techniques for the Data Magic Circle, and just let the paying customers enjoy the show….