Bottom-Up Business Intelligence

The traditional notion of data warehousing is the increasing accumulation of structured data, which distributes information across the organization, and provides the knowledge base necessary for business intelligence.

In a previous post, I pondered whether a contemporary data warehouse is analogous to an Enterprise Brain with both structured and unstructured data, and the interconnections between them, forming a digital neural network with orderly structured data firing in tandem, while the chaotic unstructured data assimilates new information.  I noted that perhaps this makes business intelligence a little more disorganized than we have traditionally imagined, but that this disorganization might actually make an organization smarter.

Business intelligence is typically viewed as part of a top-down decision management system driven by senior executives, but a potentially more intelligent business intelligence came to mind while reading Steven Johnson’s book Emergence: The Connected Lives of Ants, Brains, Cities, and Software.

Emergent systems, Johnson explained, “solve problems by drawing on masses of relatively stupid elements, rather than a single, intelligent executive branch.  They are bottom-up systems, not top-down.  They get their smarts from below.  In these systems, agents residing on one scale start producing behavior that lies one scale above them.  The movement from low-level rules to higher-level sophistication is what we call emergence.”

On a daily basis, business decisions are being made at every level of every organization.  Each of those individual decisions is analogous to an action taken by an individual ant within an ant colony.

“The colony brain,” Johnson explained, “is the sum of thousands of simple decisions executed by individual ants.”  If you’re building a system, Johnson advised, designed to learn from the ground level, a system where global intelligence and adaptability is derived from local knowledge, which I would argue is what business intelligence is trying to accomplish for the enterprise, there is an essential connection between micro-motives and macro-behavior.

Studying individual decisions, or decision makers, in isolation will not reveal the big picture of how the organization makes its business decisions.  This can only be seen by observing the entire decision management system at work in order to see the emergent behavior of business intelligence.

“We see emergent behavior in systems like ant colonies,” Johnson explained, “when the individual agents in the system pay attention to their immediate neighbors rather than wait for orders from above.  They think locally and act locally, but their collective action produces global behavior.”

“Most discussions of decision making,” Peter Drucker once said, “assume that only senior executives make decisions or that only senior executives’ decisions matter.  This is a dangerous mistake.”

“Decisions,” Drucker explained, “are made at every level of the organization, beginning with individual professional contributors and frontline supervisors.  These apparently low-level decisions are extremely important in a knowledge-based organization.  Knowledge workers are supposed to know more about their areas of specialization—for example, tax accounting—than anybody else, so their decisions are likely to have an impact throughout the company.  Making good decisions is a crucial skill at every level.  It needs to be taught explicitly to everyone in organizations that are based on knowledge.”

Senior executives provide top-down oversight (e.g., the strategic aspects of Information Governance), but a knowledge-based organization is built upon a foundation of bottom-up business intelligence.

Category: Business Intelligence
1 Comment »