Open Framework, Information Management Strategy & Collaborative Governance | Data & Social Methodology - MIKE2.0 Methodology
Collapse Expand Close

To join, please contact us.

Improve MIKE 2.0
Collapse Expand Close
Need somewhere to start? How about the most wanted pages; or the pages we know need more work; or even the stub that somebody else has started, but hasn't been able to finish. Or create a ticket for any issues you have found.

Posts Tagged ‘information theory’

by: Robert.hillard
24  Jun  2011

The “four layer” model applied to unstructured content

In my book, Information-Driven Business, I introduce a four layer model for information.  You can also read more about this model in the MIKE2.0 article: Four Layers of Information.

The four layer model provides a way of describing information in every organisation.  The model explains how information is consumed (layer 1: metrics), navigated (layer 2: dimensional), held (layer 3: atomic) and created (layer 4: operational).  Using this model helps to organisation to understand where it is overly dependent on staff or customer knowledge to manage information at any of these layers (such as summarising to report, or slicing-up in spreadsheets to answer questions).

Some people have commented that the descriptions I use in the book, and are used in the MIKE2.0 article, are geared towards structured data.  To help readers understand how the model equally applies to both structured and unstructured data, the following definitions of each layer may help

Layer 1: Metrics
For information to be used for management decision making, it ultimately needs to be summarised into a score or metrics against which “good” or “bad” can be defined.  This is the same regardless of whether we are talking about structured data or summarising a collection of unstructured content.  The metric for documents could be as simple as a count (for example, the number of policies) or a combination of factors such as the number of processes covered by a particular type of policy.

Layer 2: Dimensional or Navigational
While formally described as the dimensional layer, it is perhaps better described as the way that the organisation can be navigated.  At this layer we are talking about structuring the content in way that we can find in a systematic way (via a taxonomy).  It is from here that metrics, such as a count of policies, can be derived.  It is also from here that we go to find content in its general form (“get me all procedures associated with disaster recovery”).  For instance, in this layer policies can be cross referenced against each other.

Layer 3: Normalised or Atomic
In the unstructured sense it is better to use the term “atomic” for this layer which contains the content in its original form reference by the event that created it rather than a business taxonomy.  This layer is often handled badly in organisations but can be as simple as recording the time, author and organisational hierarchy.  It can also be aligned to business processes.  For instance, in this layer, policies and procedures should be fully formed but only associated with the scope that they are covering.

Layer 4: Operational
The fourth layer is the front line and refers to the situation and technology context in which the content is created or updated.  Examples include: social media, documents on network drives and email within the inbox of the conversation participants.  For instance, in this layer, policies are created (maybe in many parts) but have no context.

Category: Enterprise Content Management, Information Management, Information Strategy, MIKE2.0
No Comments »

by: Robert.hillard
14  May  2011

Is Information Management an evolution or a revolution?

I was recently asked by ABC Radio National in Australia to explain the principles behind my book, Information-Driven Business.  The following is extracted from the program or you can listen to the full broadcast online.

I’ve spent more than twenty years looking at how having large quantities of complex information affects every part of our lives.  Whether it is your health and welfare, or the approval of your home loan, I cannot overstate how important the recent accumulation of vast quantities of complex data about each of us has been.  Just a few years ago, who would have imagined that a company like Google would have a photo of almost every house in major cities in Australia or that so many of us would share large amounts of personal information through Facebook?

I’ve also had a life-long fascination with science and mathematics.  I’ve watched the enormous development of ideas in computing and it has often occurred to me that we are particularly bad at learning from other, more mature fields – particularly science.  The last few decades of development of information technology has seen new generations of enthusiastic entrepreneurs develop their ideas as if nothing they did had ever been thought of before.

Online shopping sites might offer great new features, but they don’t really change the idea of a traditional store or a mail-order catalogue.  Banks may be offering us new forms of credit but the idea of a home loan hasn’t really been challenged.  In both cases, businesses have evolved an existing set of business processes without really challenging who holds what information and when they hold it.

To understand the opportunity that business, consumers and societies have today, we need to go right back to the industrial revolution.  The changes then triggered an exodus of people from agriculture to manufacturing, from rural to urban settings.  Today we are in the midst of an information revolution.  This revolution is triggering a move of many jobs across global borders and the removal of many unskilled roles from the workforce.

This revolution has also seen an enormous quantity of personal information move into private hands through social media, marketing databases and also more detailed credit checks.  Previously this would have just belonged to government, or if business held it they wouldn’t have been able to compile or analyse it as it would have been written on paper buried in filing cabinets.

If you watched the science fiction movie, Minority Report, in 2002 you might have wondered at the way Tom Cruise was able to sift through data with a wave of his hand and you were probably dazzled by the wall of the store that greeted him by name and knew what his last purchase was.  Both of these innovations aren’t the vision of a distant future, rather they are here now with products like the iPad which sweeps in the same way and customised messaging in stores based on our loyalty cards or even a connection with our mobile phone.

Modern business has evolved from the industrial revolution.  The problem we face today in navigating the information revolution is that the industrial revolution taught us to use the principles of processes.  Two centuries of business has slavishly adhered to the idea that commercial and government enterprises are nothing more than the aggregate output of thousands of individual business processes.  Because no-one alive today has experienced any other form of business interaction we can be forgiven for thinking that there is no other alternative.

But we should wonder whether it is the right approach.  By learning why we have a process-oriented approach to business, we can then question whether we have really thought deeply about it.  The trigger for my doubt is that very few other things that we experience in other fields are oriented around processes.

The telephone network is very efficient at joining two people together by analysing source and destination and finding the shortest path between the two.  Social networks are very efficient as evidenced by the small number of steps to required connect anyone with anyone else on the planet and it doesn’t seem to matter what culture or society norms are in place between the two people (just think “six degrees of separation”).

Traditional business, however, relies on processes and is really very inefficient.  Just ask anyone who has ever tried to change their utility bill details or arrange a new mortgage.

Most of science and mathematics does not lend itself well to this process paradigm.  Having said that, business borrows from the small number of such examples that we do have.  Think of terms like “an idea’s half life” (borrowing from nuclear reactions to describe how new concepts need to be implemented quickly to have an impact) or “catalyst for change” (borrowing from chemistry to describe how certain actions in business have an additional and beneficial impact).  Hence, if the few examples we find already draw from the real world, then if there were more they would be likely to already appear as common business metaphors.

Of course, fields related to modern business do borrow from science.  The field of economics borrows from thermodynamics.  Modern fund managers are very familiar with using principles evolved from chaos mathematics which underpin predictive models used to support their investment in securities markets.

I argue though, that such ideas are limited to small aspects of business and by-and-large process-oriented organisations are not borrowing from science or any other field of human endeavour.  The information revolution provides the impetus to change.

Allow me to illustrate.  Consider a home loan that is written in a bank branch.  In years gone by, a paper application would have been provided to a supervisor who would have reviewed it when they had time.  If the amounts exceeded their approval threshold, they would have forwarded it upwards for further approval.  Today, this is all done electronically – but inevitably that electronic process mimics the paper one – we are continuing to use a paper process metaphor because it is what we know.  But an optimised business would have many eyes on the same application with the first available supervisor anywhere in the world being able to approve it if it met their desired criteria.

The result of the information revolution is a new paradigm.  The “information-driven business”, as opposed to the process-driven business.  This new form of enterprise is engineered around information without being tied to an individual process or activity.  Customer information doesn’t really need to be defined by how it was collected as it usually is today.  When you provide your personal details through a call centre for an electricity utility company you shouldn’t need to repeat yourself a year later when you move again and use the internet to request a service at your new address.

Once business has been freed from its process straightjacket, we have the opportunity to create a much more dynamic organization that looks more organic than artificial.  Ideas (that is products or services) can be combined or divided at will and to achieve the best possible economic or social outcome.  In a world where processes are increasingly outsourced, and physical products are being manufactured en-mass by labour efficient countries, it is incredibly important that we value information more highly than any individual transaction or process that supports the transaction.  What is more important to a department store, the completion of the sale of an individual item or the knowledge of the customer that enables the store to service him or her for many years?

My book, Information-Driven Business, takes this concept to the logical conclusion and provides methods for estimating the quantity, usability and value of information that we interact with in all aspects of commercial and government business.  These techniques apply the principles of thermodynamics through entropy, mathematics through graph theory and physics through complexity theory.  I argue that the value of modern business is largely tied-up in its intangible information.  Consider what would happen to the value of any company if you reformatted every hard disk in the organisation.

Fortuitously, at the same time that we are reconsidering the role of information in business, many branches of science are beginning to see information not as a by-product of something physical but rather the important thing in its own right.  This should be no surprise when we consider quantum mechanics of the past eighty years where we have realised that the information we retrieve about an experiment means we have adjusted the result.  Increasingly information is not a window on our universe, rather it is our universe.

The comparison between physics and business is only valid because the organisations we have built today have become so complex that it is impossible to predict in advance how they will respond to any given event or change.  What we can do, though, is predict general behaviours by looking at what information relationships exist.

In my book I provide a technique for determining the information linkages within an organisation –that is how well connected information from one part of the organisation is to information in other parts of the enterprise.

As we get better at using all of the data and information, it has greater value and market forces have naturally encouraged more of these linkages.

This has continued a long-term trend to see business consolidate and continue to get larger. However something has now changed that we need to understand.  During the past decade information standards have begun to emerge and gain acceptance.  Perhaps the most important is the eXtensible Business Reporting Language, a data standard that is used by businesses and governments around the world – including in Australia.

With information standards we are now seeing businesses decide not to merge but rather form information alliances and to trade information in real-time to provide a customer with an integrated product.  Think, for instance, about travel products and the number of individual businesses that are typically now involved in a single transaction but all appearing under the one banner. These can include the airline’s flights, an alliance airline’s connecting flights, rental cars, hotel accommodation, travel insurance and airport valet parking.  This example might seem obvious, but it has far reaching implications in the segmentation of the value chain.  For the first time in a very long time, agility is likely to drive companies to get smaller without the balancing factor of scale being needed to take an integrated product to market.

The worth of each contributing business is in how it adds information value and how it enhances the product as a whole.  With such complex partnering of many organisations, there isn’t time for a waterfall business process to form.  There is time, if we’re not careful however, for a chaotic system to form, which by its nature will provide a result which the members of the joint venture cannot predict.

In the travel example, chaos could result from the combination of different accommodation and airline loyalty schemes participating in the same product with different rules.  Can I claim hotel points if I’ve paid by redeeming airline points?  Does a credit card provider, also providing points, get access to information about the whole transaction?  With these different combinations there might be different cost and margin implications which, in-turn, cause participants to add surcharges or provide further discounts.  Does each participant need to offer the best price at every point in time, or can they discount to different channels?  Even small changes in the rules of information exchange and associated pricing can completely change whether customers are better to go via the airline, hotel or bring their constituent parts together through individual travel websites – reducing the integrated value that businesses participating in the alliance are able to provide.

With such complex relationships, the participants can do well to learn from the synergies of a biological ecosystem rather than try to predict every process pathway that a customer might choose to use.  In some cases organisms within an ecosystem can be in competition, offer each other synergies and be “food” for one another all at the same time.  Business of the twenty-first century echoes these same biological relationships.

The information revolution provides us a unique opportunity to re-invent business.  The techniques we will use to develop new approaches to everything from supermarkets to airlines will borrow less from business processes of the twentieth century and far more from physics, chemistry, biology and mathematics.  The people who are going to be best placed to invent this new world of business are the very people who have often felt most alienated from it – science graduates.

Category: Information Management, Information Strategy

by: Robert.hillard
14  Jan  2011

The Small Worlds data measure applied to business innovation

In my book, Information-Driven Business, I introduce the concept of the “Small Worlds” test on information.  In summary, this measure determines the relationship between complexity and separation in any data.  One of the best ways to apply this test is to use it to determine how innovative a new product or business idea actually is.

There are two things we can learn from the past twenty years.  The first is that new and truly disruptive businesses almost always use information in a new way (examples include the way new credit card issuers use loyalty schemes and Amazon’s ability to recommend purchases).  The second is that the information associated with truly disruptive businesses more closely adheres to the Small Worlds principle that separation and complexity have a logarithmic relationship.  That is, adding more complexity in the information only results in users having to navigate a small number of extra steps.

For instance, the telephone network of the early twentieth century was requiring a linear growth in telephone operators to keep growing, but by the second half of the twentieth century it had innovated to ensure that moving from the simplest transaction (calling next door) and the most complex (calling the other side of the world) only added a small number of exchanges.  Similarly, iTunes doesn’t just allow you buy music online, rather it innovates by reducing the number of steps required to relate information on your iPod to the artist and album that you are interested in.

You can preview through Google Books the chapter of Information-Driven Business defining the “Small Worlds” measure.

Category: Information Strategy, Information Value
1 Comment »

by: Sean.mcclowry
27  May  2009

Information Development concepts becoming more mainstream

For an interesting point of view on how how quantitative social science is becoming more making mainstream, check out Steve Miller’s great article: Hopefully we’ll see more of these methods developed in an open and collaborative fashion through frameworks like MIKE2.0.   Something we haven’t done well enough is engage the academic community becoming part of the collaborative community?

Seen any great published work on this space?  Help add it to MIKE2.0 bookmarks ..

Tags: , , ,
Category: Information Development, Information Value

by: Robert.hillard
14  Sep  2007

Single version of the truth

The concept of a single version of the truth has gained currency in Information Management to the point of being a mantra and I believe it is appropriate to introduce a few words of caution.

If I can be philosophical for a moment, Information Management theory is starting to turn the many old views of the world, including the way physics describes objects.  This isn’t a long bow to draw for Information Management professionals: Imagine a white statue in the park.  If you put on rose colored glasses, what color is the statue?  If you get everyone in the park to put on rose colored glasses, what color is the statue?  If you cover the statue in rose colored cellophane?  If you paint in statue with rose colored paint?

Survey any group and you are likely to get different answers to the color of the statue under the different circumstances I’ve outlined here.  If you at least said that painting the statue changed its color then you are admitting that it is the information (in this case color) that you receive that is important rather than what might exist in any deeper layers.

Those same rose colored glasses can apply to the enterprise data warehouse.  One version of the truth that forces everyone to see the data through rose colored glasses does not make the data rose colored!  Accountants, however, have thought long and hard about this and have rules for how you can clean-up variances that can’t be reconciled.  The most important thing is to ensure that all observers agree rather than just observe the same result, and that includes reasonable outsiders, executives and analysts.

Tags: , ,
Category: Data Quality, Enterprise Data Management
No Comments »

by: Robert.hillard
07  Aug  2007

Why is Information Management so complex?

I was speaking with a client this week who put forward the challenge that Information Management isn’t really as complicated as we in the profession make out.  I stopped for a moment to think about how I could explain the intricacy of an entire body of practice and realised that I would need to pick just one example.

Given its prominence in the industry, I decided to use Master Data Management and particularly the process of matching between sets of master data.

I started with just two lists of people (set A and set B).  I then explained how a typical algorithm would match individual records by creating a score and a threshold for matching.  No problem my client said, he could use a spreadsheet for that!

I then added a third list (set C).  Most algorithms compare two lists at a time.  That means there are three combinations: AB followed by ABC, AC followed by ACB, and BC followed by BCA.  To see why it matters, consider the following situation.

In set A, we have a record: “Robert Hillard, email robert.hillard[at]”
In set B, we have a record: “Robert Hillard, phone number +61 412 396 036”
In set C, we have a record: “Robert Hillard, phone number +61 412 396 036, email: robert.hillard[at]”

A typical business rule might require two items of data to match before the threshold is reached.  That means we need name and email, name and phone number or email and phone number to define a match.

In the first scenario we match AB first followed matching the resulting records with set C.  In this example, the two “Robert Hillard” records are not matched in the first pass meaning on the second pass when we bring in set C we can only end up with at best two records when we match the two entries to the new Robert Hillard in set C.  The final result is two instances of Robert Hillard.

In the second scenario we match AC first which results in a full match on Robert Hillard, which in turn when set B is brought in matches to the instance in that file as well.  The final result is just one instance of Robert Hillard.

Now understanding the complexity, my client tried to add a kludge solution by creating a master record for each match during an individual pass.  There isn’t enough space in this posting to explain why this doesn’t help as the number of sets increases, however suffice it to say that each such band aid solution actually adds to the complexity when more sets are added.

In summary, the more sets there are to match the more combinations there are which will affect the outcome.  For n sets there are, in fact (n-1)! (ie., n minus 1 factorial) combinations each of which will usually give a different final result for a statistically significant number of entries.  Imagine the problem facing the US government when trying to bring together lists of doctors, lawyers or other professionals across 50 state lists!

Tags: ,
Category: Information Governance, Information Strategy

Collapse Expand Close
TODAY: Tue, March 19, 2019
Collapse Expand Close
Recent Comments
Collapse Expand Close