Failure to Launch: From Big Data to Big Decisions
Forte Wares
Copyright 2014 by Forte Wares
"Big data is bullshit" - Harper Reed, President Obama Campaign CTO
Not long ago big data was all the rage, the trendiest concept in the world of business intelligence. Now, however, the hype appears to be fading fast, with few companies having realized the bottom-line benefits promised to them by various vendors from the undertaking of big data initiatives.
One of the key drivers of this gap between promise and the reality has been the shortage of effective and adequate human resources in the organizations to undertake such efforts. This vacuum has resulted in minimal actions being taken on the wealth of information big data has made available for the organizations, a core driver behind the failure of tangible impact being realized. Recent research conducted by IDG Enterprise suggests that organizations are facing numerous challenges due to this limited availability of skilled resources, with 65% indicating they are overwhelmed by the influx of data.
The key to making big data initiatives a success lies within making the produced data more digestible and usable in decision making, rather than making it just ‘more,’ resulting in the creation of an environment wherein information is used to generate real impact. Put another way, the survival of Big Data is more about making the right data (not just higher volume) available to the right people (not just higher variety) at the right time (not just higher velocity).
Pro-active Reporting
Pro-active reporting is a next generation business intelligence domain, where, instead of users reviewing standard sets of reports, solutions automatically identify and build custom reports for the users, notifying them of critical trends as well as the root causes behind such trends.
---------------------------------------
Data Unavailability: Challenger Disaster
January 28, 1986 marks a dark day in space history, with the Space Shuttle Challenger disintegrating 73 seconds into its flight, taking the lives of 7 crew members.
Years after the disaster, The Rogers Commission discovered that the vehicle was not cleared to operate at the low temperatures of that morning, and this fact was not adequately communicated to decision makers on time.
This tragic incident highlights the vital importance of pro-active reporting everywhere, as not always the right data reaches the right decision makers at the right time.
---------------------------------------
Three main pillars exist within the pro-active reporting realm:
1. An Intelligent Alert Mechanism
‘Intelligent’ triggers, notification messages, and alerts are vital components of the pro-active reporting architecture. In such architecture, rather than end users setting up alerts for re-defined thresholds on key metrics, intelligent agents continuously analyze them and discover significant changes automatically, be it foreseen or unforeseen.
The difference between the two approaches can make or break a business - while in the traditional reporting approach, a decision maker would be unknowingly at ease when the bottom line has not changed much, the decision maker following the pro-active approach would know that although the bottom line is stable, it is, for example, losing significant share in a specific market segment (temporarily balanced by an increase in share in another) and address this risk.
2. An Apt for Causal Intelligence
Another key characteristic of the pro-active reporting approach is the heavy focus on change, and the root causes behind the change. Instead of providing a traditional look at snapshot figures and charts, pro-active reports highlight delta in time, and analyze key contributors to this change. The output is usually a chain of mathematical explanations of changes - e.g. “our sales decreased by 3% this month. 90% of the decrease is attributable to the sales decrease in the youth segment for product A in region 1, which in turn was triggered due to the 10% decrease in regional advertising spend.”
Apt for Causal Intelligence is not only a matter of reporting, it is a philosophy that requires the discipline to gather and analyze data on all possible root causes of change.
3. A Readiness for What-ifs
The third important feature of the pro-active reporting approach is the focus on analyzing the impact of alternative business scenarios on the outcomes. Such scenarios can answer questions ranging from - “What if we had increased our marketing spend by 10%,” to “What if we had closed our store in downtown Manhattan,” along with sensitivity analysis on potential outcomes.
The what-if analyses should follow the same principles as causal intelligence, correlating various factors with each other, based on historical learnings.
Impact on Decision Making
Utilization of the pro-active reporting approach provides three main benefits to an enterprise:
1. Speed of Decisions
An average analysis request for a question as simple as “why did our sales drop?” can take from hours to weeks in most enterprises to answer. In addition, the question is often asked too late, after the sales has dropped and has continued to do so.
Pro-active reporting picks up the early signs of such gaps, and makes them known immediately to the right people, along with the possible root causes. This, not only decreases time to uncover the truth from weeks to seconds, but also makes it possible to address possible risks and opportunities long before it is too late.
---------------------------------------
Late Reaction: Flight 370
One of the most mysterious events in recent air travel history is the disappearance of Malaysia Airlines Flight 370. On March 8, 2014, the plane disappeared from civilian radar, with its 227 passengers and 12 crew.
One of the most striking findings about the event was the fact that air traffic controllers did not realize that the plane was missing until 17 minutes after the fact.
This case highlights the gap between speed of data collection and their interpretation by people even in most drastic events. Pro-active reporting minimizes the need for such interpretation and avoids such delays.
---------------------------------------
2. Accuracy of Decisions
The decision-making process in companies today is less than ideal in many cases, even in those that can be considered best-in-class, for a number of reasons:
The fatigue of analysts looking at same numbers over and over again results in incorrect information being reported
The lack of time to perform comprehensive analysis possibly results in key factors being overlooked
The lack of capabilities in analysts to truly uncover root causes
The interpretation of data with a bias due to various internal pressures resulting in misleading information being reported
Pro-active reporting solutions ensure the above factors that drive accuracy-related issues are bypassed, and done so in a rapid and efficient manner, analyzing all key measures as well as possible scenarios, all while ensuring zero bias is in play.
---------------------------------------
Human Oversight: The BP Oil Spill
April 20, 2010 marks one of the black pages in marine history, with 4.9 million barrels of oil spilled into the Gulf of Mexico, claiming 11 lives.
When the presidential committee reported its findings, one thing was clear - timely recognition of the risks and first signs were inadequate.
A test of the seal that resulted in the blow-out was incorrectly assessed to be in proper working order by a technical team, and the workers failed to recognize the first signs of the impending blow-out.
The committee write
s “BP did not have adequate controls in place to ensure that key decisions...were safe or sound.”
---------------------------------------
3. Cost of Decisions
Big Data analysts and scientists are some of the scarcest resources in most enterprises. Using pro-active reporting platforms frees up the valuable time of such resources, making sure that they can be used in value-added, more complex tasks that cannot be automated or done by technology in a more efficient and accurate manner.
---------------------------------------
Risk Detection: Financial Melt-Down
The 2008-2009 global financial crisis was one of the worst in financial markets history. Even the Federal Reserve was late to realize the risk, with one of the Fed’s senior forecasters recorded saying at the time: "we're still expecting a very gradual pickup in GDP growth over the next year."
The relations between trends in volumes & risks of sub-prime loans, asset-backed securities, and credit default swaps and what it meant for the market went undetected by most, with reliance instead on agency ratings to identify sound investments.
---------------------------------------
As the variety and complexity of indicators to follow increases, it becomes humanly impossible to track them over time, which is better left to systems.
Pro-active Reporting Use Cases
Application areas of pro-active reporting are as wide as regular reporting - it can be used anytime where a decision needs to be made.
Most importantly, pro-active reporting facilitates