Open Framework, Information Management Strategy & Collaborative Governance | Data & Social Methodology - MIKE2.0 Methodology
Collapse Expand Close

To join, please contact us.

Improve MIKE 2.0
Collapse Expand Close
Need somewhere to start? How about the most wanted pages; or the pages we know need more work; or even the stub that somebody else has started, but hasn't been able to finish. Or create a ticket for any issues you have found.

Posts Tagged ‘data science’

by: Robert.hillard
26  Feb  2016

New ideas aren’t always brilliant

Change is the lifeblood of organisations.  It is essential in our products, technology, organisational models and every aspect of how we work and produce for the benefit of our stakeholders and ourselves.  Everyone can think of organisations that failed to change quickly enough, perhaps the best example being Kodak.

The virtual shelves are full of books written through the eyes of the executive who is trying to make change happen.  In almost every case, the assumption is that the change proposed is the right one and that anyone opposing change is a negative for the business.  The persona of the obstacle is all too often an aging middle manager who is stuck in their ways and unwilling to embrace modernity.

Everyone is keen to denounce doubters of new ideas as being stuck in their ways.  However, few also point to all of those bad ideas that weren’t guiding the organisation to a better future but rather were just genuinely bad ideas!  Maybe if there had been a few more obstacles to change we might still have companies like Pan Am and HIH in Australia.

How do you know whether someone’s objection to change is recalcitrance and when it is a genuine insight that the change is a bad one?

Seek feedback

Everyone is capable of producing ideas but most great ideas are recognised only after they are tested in the crucible of the real world.  No one, regardless of how smart they are, comes up with something brilliant every time.  For every “hit” there are multiple “misses” which looked just as good when written down but fail the same real world test.  While the best leaders are better than most at recognising the hits and misses, no one is able to spot them all.

It sounds obvious, but it is important to seek feedback from others no matter how confident you are in the approach you want to take.  It doesn’t matter if it is an organisation structure, new product or a marketing campaign.

Feedback may not come from the obvious places and the challenge is to look for it from people who have insight into unintended consequences.  This is more important than ever as our organisations and products have become more complex.  I’ve looked before at why our organisations don’t operate the way we expect when I asked why I aren’t I working a four hour day.  I’ve also suggested that it is important to make simplicity a goal in its own right, but recognise that it is a goal that will never be reached (see Trading your way to IT simplicity).

Testing the ideas

Beyond feedback, there are two ways that a leader can test their ideas.  The first is by evidence and the second is by debate.

Testing by evidence requires rigour.  While it is tempting to make the available data fit the hypothesis, it really only works if an experiment is designed in advance with two distinct outcomes, the status quo (or “null hypothesis”) and the alternative which represents the proposed change.

In almost every case of business transformation, new product or investment, it is hard to define experiments.  The advice that has come through observing companies like Netflix and Capital One is that investing in designing tests is extremely effective no matter how difficult it seems.

Another method that leaders can apply is the running of short, sharp, debates.  Recruit six candidates from a variety of backgrounds and randomly assign them to argue for and against the proposal.  The debates work best with an enthusiastic audience who are instructed to vote for the best argued position (and not just for the idea they like).  Of course, the strongest arguments are usually based on the best available data!

A great leader will listen to feedback and be prepared to walk away from change that doesn’t stand up to testing or debate.  When these leaders do commit to change, they are seen to have taken a considered approach and will not only lead their organisations further but they will keep them at their destination for longer.

Tags: , ,
Category: Enterprise2.0
No Comments »

by: Ocdqblog
30  Apr  2013

Bigger Questions, not Bigger Data

In her book Being Wrong: Adventures in the Margin of Error, Kathryn Schulz explained “the pivotal insight of the Scientific Revolution was that the advancement of knowledge depends on current theories collapsing in the face of new insights and discoveries.  In this model of progress, errors do not lead us away from the truth. Instead, they edge us incrementally toward it.”

In his book Ignorance: How It Drives Science, Stuart Firestein explained “questions are more relevant than answers.  Questions are bigger than answers.  One good question can give rise to several layers of answers, inspire decades-long searches for solutions, generate whole new fields of inquiry, and prompt changes in entrenched thinking.  Answers, on the other hand, often end the process.”

Unfortunately, some people seem to misunderstand the goal of big data and data science to be the pursuit to provide the answers to all of our questions.  Some go so far as to claim that eventually we will know everything, that soon we will be able to foretell the future with absolute certainty.

These were a few of the misunderstandings addressed by Andrew McAfee in his recent Harvard Business Review blog post Pundits: Stop Sounding Ignorant About Data. “I’ve been talking and hanging out with a lot of data geeks over the past months and even though they’re highly ambitious people,” McAfee concluded, “they’re very circumspect when they talk about their work.  They know that the universe is a ridiculously messy and complex place and that all we can do is chip away at its mysteries with whatever tools are available, our brains always first and foremost among them.  The geeks are excited these days because in the current era of Big Data the tools just got a whole lot better.”

“The right question asked in the right way, rather than the accumulation of more data,” Firestein concluded, “allows a field to progress.  Scientists don’t just design an experiment based on what they don’t know.  The truly successful strategy is one that provides them even a glimpse of what’s on the other side of their ignorance and an opportunity to see if they can’t get the question to be bigger.  Ignorance works as the engine of science because it is virtually unbounded, and that makes science much more expansive.”

Science has always been about bigger questions, not bigger data.

Tags: , ,
Category: Information Development

by: Ocdqblog
28  Mar  2013

A Contrarian’s View of Unstructured Data

“If you analyzed the flow of digital data in 1980,” Stephen Baker wrote in his 2011 book Final Jeopardy: Man vs. Machine and the Quest to Know Everything, “only a smidgen of the world’s information had found its way into computers.”

“Back then, the big mainframes and the new microcomputers housed business records, tax returns, real estate transactions, and mountains of scientific data.  But much of the world’s information existed in the form of words—conversations at the coffee shop, phone calls, books, messages scrawled on Post-its, term papers, the play-by-play of the Super Bowl, the seven o’clock news.  Far more than numbers, words spelled out when humans were thinking, what they knew, what they wanted, whom they loved.  And most of those words, and the data they contained, vanished quickly.  They faded in fallible human memories, they piled up in dumpsters and moldered in damp basements.  Most of these words never reached computers, much less networks.”

However, during the era of big data, things have significantly changed.  “In the last decade,” Baker continued, “as billions of people have migrated their work, mail, reading, phone calls, and webs of friendships to digital networks, a giant new species of data has arisen: unstructured data.”

“It’s the growing heap of sounds and images that we produce, along with trillions of words.  Chaotic by nature, it doesn’t fit neatly into an Excel spreadsheet.  Yet it describes the minute-by-minute goings-on of much of the planet.  This gold mine is doubling in size every year.  Of all the data stored in the world’s computers and coursing through its networks, the vast majority is unstructured.”

One of Melinda Thielbar’s three questions of data science is: “Are these results actionable?”  As Baker explained, unstructured data describes the minute-by-minute goings-on of much of the planet, so the results of analyzing unstructured data must be actionable, right?

Although sentiment analysis of unstructured social media data is often lauded as a great example, late last year Augie Ray wrote a great blog post asking How Powerful Is Social Media Sentiment Really?

My contrarian’s view of unstructured data is that it is, in large part, gigabytes of gossip and yottabytes of yada yada digitized, rumors and hearsay amplified by the illusion-of-truth effect and succumbing to the perception-is-reality effect until the noise amplifies so much that its static solidifies into a signal.

As Roberta Wohlstetter originally defined the terms, signal is the indication of an underlying truth behind a statistical or predictive problem, and noise is the sound produced by competing signals.

The competing signals from unstructured data are competing with other signals in a digital world of seemingly infinite channels broadcasting a cacophony that makes one nostalgic for a luddite’s dream of a world before word of mouth became word of data, and before private thoughts contained within the neural networks of our minds became public thoughts shared within social networks, such as Twitter, Facebook, and LinkedIn.

“While it may seem heretical to say,” Ray explained, “I believe there is ample evidence social media sentiment does not matter equally in every industry to every company in every situation.  Social media sentiment has been elevated to God-like status when really it is more of a minor deity.  In most situations, what others are saying does not trump our own personal experiences.  In addition, while public sentiment may be a factor in our purchase decisions, we weigh it against many other important factors such as price, convenience, perception of quality, etc.”

Social media is not the only source of unstructured data, nor am I suggesting there’s no business value in this category of big data.  However, sometimes a contrarian’s view is necessary to temper unchecked enthusiasm, and a lot of big data is not only unstructured, but enthusiasm for it is often unchecked.

Tags: , , , , , ,
Category: Information Development

by: Phil Simon
18  Nov  2012

Curiosity, Big Data, and Data Science

If you haven’t heard of data science, you will soon. As organizations realize that Big Data isn’t going away, they will finally come around. This is always the case with the technology adoption lifecycle. Yes, this may very well mean new hardware purchases and upgrades, as well as new software solutions like Hadoop and NoSQL. At some point, however, employees with new skills will have to make all of this new stuff sing and dance.

Enter the Data Scientist

Part statistician, part coder, part data modeler, and part business person, the term has grown in importance since being introduced in 2008. But I’d argue that the single most important attribute of the data scientist is a childlike curiosity of why things happen–or don’t.

In their 2010 HBR piece “Data Scientist: The Sexiest Job of the 21st Century“, Thomas H. Davenport and D.J. Patil write about the key characteristics of these folks. From the piece:

But we would say the dominant trait among data scientists is an intense curiosity—a desire to go beneath the surface of a problem, find the questions at its heart, and distill them into a very clear set of hypotheses that can be tested. This often entails the associative thinking that characterizes the most creative scientists in any field. For example, we know of a data scientist studying a fraud problem who realized that it was analogous to a type of DNA sequencing problem. By bringing together those disparate worlds, he and his team were able to craft a solution that dramatically reduced fraud losses.

Think about what a real scientist does for a moment. Whether trying to invent a drug or cure a disease, scientists look at data, form hypothesis, test them, more often than not fail, reevaluate, and refine. Many problems remain unsolved even after years of analysis. Louis Pasteur didn’t create a vaccine for rabies and anthrax over a weekend. Science is not a linear process.

Simon Says

I’d argue that the same thing applies to data science. Detecting patterns in datasets in the petabytes much different than writing a simple SELECT statement or doing ETL. Big Data means plenty of iterations and failures. Understanding why sales are slipping or customer behavior in general are not simple endeavors–nor are they static. That is, factors motivating people to buy products and services will probably change over time. Paying a data scientist a king’s ransom may come with the expectations of immediate, profound insights. That may well be the case, but it’s also entirely plausible that progress will take a great deal of time, especially at the beginning.

As Jill Dyche points out on HBR, there’s rarely a eureka moment. If your organizational culture does not permit failure and insists upon immediate results, maybe hiring a data scientist isn’t wise. Your organization should save some money and revisit Big Data and data science in five years. That is, if your organizational is still around.


What say you?

Tags: , , ,
Category: Information Strategy, Information Value
No Comments »

Collapse Expand Close
TODAY: Wed, March 20, 2019
Collapse Expand Close
Recent Comments
Collapse Expand Close