The past decade has seen profound changes in the data world. The process of storing and querying unthinkably huge amounts of data has become cheap, fast, and easy. But just because companies can use big data, doesn’t mean they know what to do with all this newfound power.
Buying a 400hp sports car doesn’t do you much good if you’re driving on the same sleepy country road as before. And it can actually be detrimental if you don’t know how to handle all that power.
Because the hype cycle, as usual, has gotten ahead of the reality, lots of people have been talking “Big Data” without actually realizing much value from the data they’re storing.
Source: Google Trends – Big Data
Thankfully, that’s all beginning to change. More and more companies are finding new and exciting ways to leverage their investments in modern data warehouses. An interesting thing to note about the three big shifts driving this transformation is that all of them involve technology, but are fundamentally driven by human skills.
The first big shift is that more and more companies are moving past historical data analysis. Data isn’t just used in the rear-view mirror.
These companies are embracing operational analytics.
This means providing near-real-time data access that helps employees do their jobs better.
For most of its history, business intelligence was entirely backward looking. You’d make a decision, wait for it to have an impact, and then examine the data to see its effect. But as faster databases and more efficient data pipelines have become the norm, the smartest companies have shifted to new ways of working.
Now they can put the right data in the right hands at the right time so that it actually informs the decision, rather than just providing retrospective feedback on how the decision turned out. It’s the difference between being able to map current traffic and being able to reroute around a traffic jam.
The second shift is driven directly by the first: the death of the HiPPO. HiPPO is an acronym for the Highest Paid Person’s Opinion, and until recently, that was the primary way that decisions were made in the enterprise.
The game changer? Readily accessible data.
Now, rather than spending the meeting trying to convince the boss that you’re right, you can bring your analysis and respond to questions and challenges in real time. If someone wants to dig deeper into the analysis that led to your conclusions, you can do that in real time, rather than telling them you’ll get back to them.
The organizations that are seeing the most benefit from these changes are the ones where the leaders embrace being proven wrong by data. When employees see that even executives are willing to be humble in the face of contrary evidence, they realize that data-informed decision-making will advance their careers and that clinging to an idea in the face of data that debunks it will be penalized.
The final change that the database revolution is driving is one of breaking down the data siloes that have hampered business decision-making for too long. Every business system today produces its own stream of data, but the expensive, complex process of moving those disparate data sources to a single location and transforming them so that they can be joined has been prohibitive until recently.
Now, all those data sources can be piped into a data lake or a cloud data warehouse with relative ease. Once they’re colocated, modern data tools make it easy to transform the data, so you can easily find linkages between data from different sources.
This might seem like a relatively trivial change from the old world, but the implications are huge.
In the past, it would have been very difficult to see how prospects who come in via different marketing channels perform as customers because that analysis requires bringing together marketing, sales, transactional, support, and account management data, at a minimum.
Similarly, it was hard to see how digital ads affect customer loyalty because that entails analyzing ad spend data, web traffic data, and transactional data all together. Understanding how shipping speeds affect customer satisfaction would also have been difficult because it requires transactional data, logistical warehouse data, delivery data from shippers’ application program interfaces (APIs), and the results from customer satisfaction surveys.
This type of big-picture analysis—the kind that can be accomplished only by knitting disparate data together—leads to huge shifts in company strategy. While they may have been possible before, they weren’t easy or fast. Now, with the advances in the underlying technology, they can be accomplished in minutes or hours, rather than days or weeks.
As often happens with technological advances, the impacts of the profound shifts in database technology have taken some time to flow out to businesses. But now that the lag time has passed, enterprises are starting to rethink the precise ways that data can drive their businesses. As usual, we can expect these shifts to accelerate as companies further adapt to this new world.