With every new technology and management trend, it’s important to examine whether it’s working after a few years. Some, like business process reengineering and Bitcoin, didn’t have a successful first few years. Others, like smartphones and ERP, have been successful enough to prosper for years to come.
The fifth annual Big Data Executive Survey from NewVantage Partners found that most big data initiatives have been successful. Big data was defined broadly for the survey as approaches that enable organizations to analyze any and all data with greater speed and agility. Datasets may be large or small, legacy sources or new sources, structured or unstructured. In the survey, which involved business and technology leaders from 50 leading companies, 48.4% of the respondents said that their firms have achieved “measurable results” from their big data investments. A higher percentage, 80.7%, felt that their big data initiatives have been successful. Only 1.6% said their efforts were a failure; for some it’s still too early to tell.
This is great news. Big data programs aren’t cheap—37% reported that their organization has invested more than $100MM on big data initiatives, and 6.5% spent over $1B—so it’s wonderful that they are paying off. Big data has been fairly high on the hype-o-meter, so it’s also good to see that there is real substance to it. The companies in the survey are generally early adopters of new technologies (95% said they’d undertaken a big data initiative over the past five years), but it bodes well for smaller and more conservative firms that the early work on big data is going well. The respondents (mostly financial services giants) are heavy users of data and at the forefront of adoption when it comes to experimenting with data and analytics capabilities through centers of excellence, big data labs, analytics sandboxes, and the like.
This doesn’t mean, of course, that firms haven’t encountered some obstacles along the way. The primary impediment to big data progress seems not to be hardware or software, but “wetware”—human and organizational factors (Figure 1). 43% of surveyed companies mentioned “lack of organizational alignment” as an impediment. 41% pointed specifically to middle management as the culprit; the same percentage faulted “business resistance or lack of understanding.” 29.5% said their organizations lacked a coherent data strategy. 86% say their companies have tried to create a data-driven culture, but only 37% say they’ve been successful at it.
|Figure 1 – Cultural Impediments to Big Data Business Adoption|
|Insufficient organizational alignment||42.6%|
|Lack of middle management adoption and understanding||41.0%|
|Business resistance or lack of understanding||41.0%|
|Lack of a coherent data strategy||29.5%|
|Technology resistance or lack of understanding||27.9%|
|Inability to create a shared vision||26.2%|
|Lack of data governance policies and practices||21.3%|
And some objectives have been more likely to be achieved than others (Figure 2). Not surprisingly, cost reduction projects are the most likely both to have been started (73%) and successfully achieved (49%). Other objectives included “creating new avenues for innovation and disruption” (65% started, 44% successful), speeding up new capabilities and services (65% and 31%), and launching new products and services (63% and 36%). Perhaps the most ambitious objectives involve monetizing big data (55% and 33%) and generally transforming the business (52% and 28%). We are actually amazed that even these highly ambitious objectives have been achieved by a third and a quarter of the firms responding.
The survey also provides a bit of sobering context about the big data efforts of big companies and a sense of what might be coming next. Most of these executives clearly feel that some big bets and transformational outcomes are necessary, because 47% feel that major disruption is coming to their industries over the next decade. Large financial services firms were particularly concerned about disruption—presumably from “fintech” firms and startups as well as from large established firms like Google, Amazon, and other big data leaders.
What will follow big data as the next disruptive technology? Signs point to cognitive technologies. 44% said the technology would impact their firm over the next decade, and 69% of those had already begun working with cognitive. Digital (mobile/social/Internet of Things) technologies were rated second most disruptive, but 78% said they were already working on them. Fintech was highly ranked as a disruptive force by financial firms, but obviously less so by those in other industries.
So this is no time for even the most sophisticated firm to rest on its big data laurels. New technologies and new ways of using them in business will undoubtedly emerge. But it’s nice to know that at least one set of technologies appears to have paid off for its users.
Randy Bean is Founder and CEO of NewVantage Partners, thought-leaders and management consultants in innovation through Big Data and emerging capabilities. You can follow him at @RandyBeanNVP.
Tom Davenport, the author of several best-selling management books on analytics and big data, is the President’s Distinguished Professor of Information Technology and Management at Babson College, a Fellow of the MIT Initiative on the Digital Economy, co-founder of the International Institute for Analytics, and an independent senior adviser to Deloitte Analytics. He also is a member of the Data Informed Board of Advisers.
Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise, plus get instant access to more than 20 eBooks.