As decision makers, none of us is a rational actor, meaning we all make wrong decisions or have inaccurate or incomplete information regarding events and facts.
Cognitive bias is a subject explored frequently in my blog on decision making and refers to our tendency to make illogical decisions, based on inaccurate or incomplete data, that satisfies an emotional need or the desire to be “right.” We are all subject to cognitive bias, without being aware that it is occurring.
User-driven data exploration provides a unique opportunity to eliminate cognitive bias in decision making and thought processes. In the beginning of this chapter, we pointed out an inherent aspect of the mind: It is best suited for having ideas, not holding and processing information. This fundamental nature of the mind in part explains our tendency toward cognitive bias; we simply cannot retain enough information to eliminate cognitive bias. With Endeca Studio and its capability to allow average business users to quickly create applications and examine data, cognitive bias can be replaced by informed, fact-based decision making and idea formation.
In the previous example, you looked at airline delay information. Most of us believe that weather-related delays are the most commonly occurring delays because most of us have been stuck at an airport waiting on a storm to pass or had to wait on an aircraft to be deiced. Yet, when you examine the airline delay information, weather-related delays are insignificant in terms of their frequency of occurrence.
For most of us, the financial troubles occurring from 2008 to 2010 would lead us to form an opinion that the worst-managed bank failures occurred during this time period. Yet the data you examined on FDIC bank failures debunks this idea. While 2009 was a bad year in terms of bank failures, the largest single bank failure occurred in 1989, and the scope and breadth of the late-1980s savings-and-loan crisis resulted in more bank failures with large losses. This tendency to focus on the most recent events is often called recency bias.
User-driven data exploration is a valuable tool for building a wallet or portfolio of data that produces useful facts and knowledge. User-driven data exploration, in part, feeds enterprise data exploration to provide data sources that are combined with other data sources in enterprise data exploration. You will explore these tools and methods in the next blogs.
