This article was written by Snowflake and originally appeared on the Snowflake Blog here: https://www.snowflake.com/blog/forging-a-truly-data-driven-organization/
In a 2020 study performed by Nature Research, 70 different teams of neuroimaging experts were asked to test nine hypotheses by looking at the same MRI data set. You may not be surprised to learn that these teams reached a wide range of different conclusions, in part because no two teams chose identical workflows to analyze the data.
With seventy teams, there were 70 different workflows.
The exercise illustrates a vital point about many kinds of tasks: Having data doesn’t automatically mean agreeing on conclusions or a course of action.
Over the past decade or so, a growing number of companies have voiced the goal of becoming data-driven organizations. There is plenty of incentive—studies and anecdotal evidence alike show the enormous potential power of business analytics. Snowflake’s product and service offerings are built on that foundation, aimed at providing seamless data collaboration.
But in addition to powerful tools, many organizations also need help with the people and process aspects of data analysis. This post is the introduction to a series dedicated to helping readers make strides towards becoming truly data-driven. To kick off, let’s define the goal clearly, describe the potential payoff, and lay out some of the hurdles to be overcome.
WHAT DOES “DATA-DRIVEN” REALLY MEAN?
Plenty of definitions of “a data-driven organization” are circulating. “Just because you have collected a pile of data does not make you data-driven,” said Kent Graziano, Snowflake Chief Technical Evangelist. “That’s just data hoarding.
“Being data-driven is a mindset that must be embraced throughout the organization if you want to get the maximum value from all that data,” Graziano said.
Rather than trying to zero in on the “best” option, it is perhaps more important to focus on a definition that is useful. With that in mind, here is a working definition for the purposes of this series of blog posts:
A data-driven organization continually improves its ability to access and analyze data consistently to make accurate decisions.
Each element of this definition is purposeful—continual improvement, growing access, consistent analysis, and accurate decisions—and future posts will circle back to it as a lodestar.
THE DIFFERENCE DATA CAN MAKE
The benefits of being data-driven have been measured in many ways.
A seminal Harvard Business Review article in 2012 claimed companies that emphasize data-informed decision-making outperform their peers by 5% in productivity and 6% in profitability.
A 2018 Forrester Research report suggested that this kind of advantage continues today, describing “insights-driven businesses…growing at an average more than 30% annually.”
Other studies and publications such as Wired have called data “the new oil” for the 21st century and Dataversity called data the foundation for digital transformation. And many point out well-known examples such as Netflix’s content recommendation engine or GE’s “digital twin” manufacturing simulations to emphasize the point.
Of course, the level of analytic excellence that outperformed the market three or more years ago may now be just table stakes. That’s why, for this series of posts, our working definition of the data-driven organization doesn’t focus on an end state. It’s about ongoing improvement.
IDENTIFYING AND OVERCOMING THE OBSTACLES
Improving analytical performance isn’t just a matter of gathering more data, although that can certainly lead to breakthroughs.
Common hurdles that your company might face include:
- Data literacy. According to Dataversity, a survey by the Data Literacy Project found 76% of business decision-makers lack confidence in their own ability to correctly understand and use data.
- Competition for analytical talent. In 2018, the U.S. Bureau of Labor Statistics predicted that demand for statisticians, who can “apply statistical theory and methods to collect, interpret, and summarize data,” would grow 33% by 2026.
- Dirty or untrusted data. This is an old problem that persists today—for example, a 2020 survey conducted by ObservePoint and reported by Business Ahead found 88% of marketers don’t trust their website analytics data. In another recent survey conducted by Anaconda and reported by Datanami, data scientists estimated 45% of their time is spent cleaning and prepping data, rather than analyzing it for insights.
- Cognitive bias. Humans are subject to well-known challenges in evaluating data, including availability bias, confirmation bias, and gambler’s fallacy. Economic psychologist Daniel Kahneman’s book Thinking, Fast and Slow provides a useful breakdown of many such mental mechanisms.