I saw a headline this week that read, “Big Data is Out; Machine Learning is In.” The post was based on Gartner’s “hype cycle,” which showed while big data had big hype in 2014, it has all but fallen off the “hype map” for 2015.
Does that mean I should start looking for a new job?
I hardly think so.
As the article pointed out, perhaps the reason that the hype around big data is fizzling is because the technologies that were all the big news last year are already in use and no longer big news. But that doesn’t mean that big data is no longer relevant. The fact of the matter is that the technologies that did make Gartner’s 2015 hype cycle chart – including autonomous vehicles, machine learning, and the Internet of Things – all produce and rely on large quantities of data.
If anything, our reliance on big data will increase, not decrease.
This isn’t the first time that the media has prematurely proclaimed the demise of big data. After an article in the journal Science last year showed that Google’s much-touted big data experiment to predict flu trends was massively overestimating the number of flu cases, headlines popped up all over declaring that big data was a big mistake.
But let’s be clear: Big data is not responsible for bad analytics, and big data is here to stay.
In its purest form, big data is just that: data.
Even a cursory examination of the mind-boggling statistics that surround big data today reveal that we are positively swimming in data, and that’s not going to change:
- More data has been created in the past two years than in the entire previous history of the human race.
- By the year 2020, about 1.7 megabytes of new information will be created every second for every human being on the planet.
- By then, our accumulated digital knowledge will be around 44 zettabytes, or 44 trillion gigabytes, up from just 4.4 zettabytes today.
The problem is not the data. The problem is that our ability to properly analyze that data and draw conclusions from it is not keeping pace with the rate at which we create it.
And that’s OK. We will catch up. Because I have seen what big data and the associated data analytics can do for business, I know that the talent and resources will be poured into this field and our abilities will improve at a prodigious rate.
That’s why I’m not worried about whether the hype around big data is on the rise or the decline.
Remember the buzz around e-commerce or business intelligence? They all received overinflated valuations and made promises they couldn’t keep, but the phenomena themselves became very real and today are part of our daily lives and business.
So it will be – so it already is – with big data. It may be simply that the hype is dying down because vast quantities of data are becoming commonplace. Even the smallest businesses run by a single person can access and analyze data about their business that were not available to them just five years ago.
At this point, only 0.5 percent of the data we create is ever analyzed and used. Imagine the potential that exists in even another fraction of that information? By some estimates, just a 10 percent increase in data accessibility will result in more than $65 million additional net income for a typical Fortune 1000 company.
Those aren’t numbers any company can afford to ignore. And that’s why I believe that big data will persist, whether it’s hyped up or not.
The data isn’t going anywhere, and someone is going to figure out how to put it to good use. The only question is which businesses will get there first to reap the most reward.
Bernard Marr is a bestselling author, keynote speaker, strategic performance consultant, and analytics, KPI, and big data guru. He helps companies to better manage, measure, report, and analyze performance. His leading-edge work with major companies, organizations, and governments across the globe makes him an acclaimed and award-winning keynote speaker, researcher, consultant, and teacher.
Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise.