Collecting Data in Real Time, but Understanding It in Stale Time

Collecting Data in Real Time, but Understanding It in Stale Time

We have yet to meet an organization that has told us it has a serious data collection problem, but we’ve heard from countless organizations that they can’t understand the mountains of data they collect. If you listen closely to most organizations’ data challenges, they will admit they are data-rich and information-poor. In other words, they have a real-time data collection strategy, but they only understand their data in stale time.


With this in mind, we propose this simple equation as a guiding principle when reading this book:

data collection
data understanding
= the price of not knowing

Take a look at Figure 1. A typical organization’s ability to collect data is illustrated by the steeply sloped thick line. Over time, lots and lots of data is collected, and the speed at which data is generated increases, often exponentially. Meanwhile, the organization’s data understanding capabilities grow more slowly, as illustrated by the thinner, flatter line.

Now take all the space between the thick line (data collection) and the thin line (data understanding), and you have what we call the price of not knowing. In this gap, the organization is guilty of not knowing what it could already know (or may have known in the past). The consequence? You name it: money lost, opportunities squandered, vacation flights missed, cars damaged, fraud enabled, lives lost, and more. We sometimes jokingly refer to this as Enterprise Amnesia.

Graphical representation of a typical organization’s data

Figure 1. Graphical representation of a typical organization’s data collection and data understanding capabilities

While you can argue that organizations simply don’t have the ability to understand the impact of the data coming in (this book will change that), what about the “things” they used to know but have forgotten? That’s context. We’ve all experienced this in our personal lives. For example, when your favorite airline delays your plane for the third time in a month, but the customer service agent who is rebooking your flight has no idea what you’ve been through as they work on your case, that’s a lack of context. Imagine if the agent proactively apologized and upgraded you on your return flight. Here is a missed opportunity for a great client interaction because the airline, and the agent, lacks context. They fail to provide empathy, because they literally have no idea this is the third time their operations have altered your flight plans...but they do know it...but it’s not well known enough for anyone to act on it (the context is forgotten).

Consider all the events any company already knows about, or could know about. Do they apply this knowledge to a 24/7 decisioning environment? For example, an electrical power company knows what a compromised power tower looks like and understands its characteristics: perhaps a blown transformer, rusting bolts that support the infrastructure, or encroaching brush and trees that might catch fire. This same provider has also likely recorded the impact that rainfall and salinity have on its infrastructure. But has it turned that recorded knowledge into simulations, to predict when a tower needs proactive maintenance? Does this company have a static time-based maintenance protocol or is it using conditioned-based monitoring to trigger maintenance routines? Does it use drones with computer vision to rapidly and more safely inspect those power towers? If the answer is “no,” these are signs of Enterprise Amnesia.

If organizations start applying data acumen (a term we’ll use to include AI, machine learning, and deep learning, as well as other approaches we discuss in the book), they can generate a new data collection curve (the dashed line in Figure 2). This curve will capture some of the value hidden in the amnesia abyss (the areas between the thick and thin lines) and open up new opportunities for top-line growth, better service, and better outcomes.

 Organizations that apply data acumen


Figure 2. Organizations that apply data acumen will notice a greater opportunity to correlate data collection and data understanding

Notice that while this new dashed data collection curve is sloped more steeply, it’s not a straight line: the curve has humps, lumps, and bumps. Modernizing your approach to data, and thus generating a new data collection curve, is a highly agile process that will encounter failures, success, and restarts.


Вас заинтересует / Intresting for you:

Healthcare Analytics Use Case ...
Healthcare Analytics Use Case ... 1078 views Stepan Ushakov Tue, 17 Sep 2019, 06:09:25
Importance of Data Science
Importance of Data Science 823 views Дэн Sun, 17 Jun 2018, 06:44:06
Data Science and Big Data
Data Science and Big Data 746 views Дэн Sat, 16 Jun 2018, 17:53:54
Future Trends in Analytics: Mo...
Future Trends in Analytics: Mo... 282 views Даниил Tue, 29 Oct 2019, 06:42:44