The hype surrounding big data has reached a fever pitch, with analysts, journalists, and, of course, software providers heralding big data as the omnipotent solution to nearly every business challenge. But, has it truly reached “revolutionary” status? The answer is no, and here’s why.
There is no dispute that we live in a data-filled world. In 2012, we created 2.8 zettabytes of data, a number that’s predicted to skyrocket to 44 zettabytes by 2020. This massive data explosion is driving a strong demand for data scientists – experts who are skilled in analyzing and making sense of this overwhelming cascade of information. In fact, some 4.4 million new IT jobs are expected to be created globally by 2015 to support the big data explosion, with nearly half those in the United States alone. Instead of encouraging their children to become doctors or programmers, mothers today are telling their children to study data science.
We have seen this particular phenomenon before: when computers first emerged, everyone was encouraged to become a programmer; and as the Internet took shape, aspiring Web developers flooded into digital design programs at technical schools and universities across the country.
But the truth is that the PC revolution didn’t start with the deployment of mainframes in the 1950s. It began in the 1980s, when Apple released the Mac 2, making a desktop computer accessible to the average user. Steve Jobs recognized that people didn’t want to build a home PC from a kit, as he’d originally thought. Instead, they wanted a fully-functional PC that they could switch on and start using. Similarly, the Internet revolution didn’t start with the modem, or with a stampede of freshly trained Web developers. It started with Netscape, which brought the Internet into the homes of users around the world, making it accessible to the masses.
The problem with this so-called big data “revolution” is that we are not quite there yet. While the world is certainly data filled, it’s not yet data driven. The data is there at our disposal, but it has not yet been made fully accessible to the masses. In fact, one-third of business leaders say they are still so uncertain about the accuracy of their analysis that they don’t trust it to make decisions. How can something that’s so untrustworthy be considered “revolutionary?”
In order to experience a true revolution, we must bring the power of big data and analytics to the masses. We need software we can trust that enables business users – not just data scientists – to examine, explore, and act on this powerful data. In the same way that Microsoft Word put the power of word processing in the hands of every person in the office, the true big data revolution will happen only when we can put the power of analytics into the hands of every business user.
To reach that watershed moment, we must completely rethink the way we approach data analysis. The conventional hypothesis-based method, which has changed very little since ancient Greece, still requires a human to formulate and ask a question, and then design a query to find the answer within the data. But this method is built upon the assumption that the human knows the right questions to ask in the first place. Ask the wrong questions, and you will get the wrong answers. With the massive volume, variety, and velocity of data available today, this approach represents a fundamental flaw in the way we conduct analysis. There are millions of potential questions we could ask of our data. But people don’t know which ones they should ask. For example, “Are we selling more to women? To teenagers? In Boston? How about to teenagers in Boston?” We need software that can ask all of the questions and surface the patterns we need to see, whether we know it or not.
Business users don’t want to become data scientists. But they do want to leverage data science to impact their revenue, costs, and risks. The lightning-fast pace of data accumulation and change means that current data is outdated in mere minutes. The scarcity of data scientists means relying on manual analysis is too time consuming and expensive. A real big data revolution is made possible only by real-time, self-service, dynamic analysis that gives business users the answers to every question in the time span of a coffee break. And truly revolutionary solutions will go a step further, to explain the answers and give users an opportunity to easily overlay their own human instincts and domain knowledge to further enhance the analysis.
By making this advanced analysis readily accessible for the masses, we can finally realize the promise of the big data revolution: analysis that can be done by any business user without training, on their own data, without any customization, in just five minutes or less.
Before founding BeyondCore, Arijit Sengupta held a variety of technical and management positions at Oracle and Microsoft. Arijit has been granted a dozen patents in advanced analytics, business process as a service, operational risk, privacy and information security. Arijit has guest lectured at Stanford, spoken at conferences in a dozen countries, and was written about in The World Is Flat 3.0, The New York Times, San Jose Mercury News, Harvard Business Review and The Economist. Arijit holds an MBA with distinction from the Harvard Business School and Bachelor’s degrees with distinction in Computer Science and Economics from Stanford University.
Home page image via Wikipedia and used under Creative Commons license.
Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise.