With analytics the vogue concept in modern business, surely enterprises everywhere are integrating large-scale analytics into their operations. Surely big data has gone live.
Lavastorm, a Massachusetts-based analytics software firm, revealed a far more restrained commitment to big data when it announced the results of its Analytics 2014 survey. Despite widespread investments by companies and almost constant coverage of big data in business media, 75 percent of businesses have yet to successfully put big data analytics solutions into production.
Almost 500 executives, analysts, and researchers responded to the survey. Their feedback shows some of the major obstacles that enterprises meet when trying to make data actionable. Lavastorm’s John Joseph, vice president of Product Marketing, sat down with Data Informed to discuss the survey as well as the challenges companies face when putting big data theories to work, where data pros are learning their skills, and the growing focus on data quality.
Data Informed: Did you anticipate a large number of respondents who haven’t yet put big data into production?
John Joseph: Analytics is a very vibrant space. It’s also a very complicated space. There’s a lot going on with different kinds of analytics. We wanted to know what the major trends were, how people were approaching analytics within their companies, whether analytics was still a growing segment of their budget, how they were going to allocate that money if there was increased investment. We wanted to understand where people were in their maturity of using big data and actually deploying it.
What struck you about big data’s absence from actual business operations?
Joseph: The biggest challenge overall for people was actually turning insights into action. If you think of a dashboard, a dashboard highlights some thing, and there’s a discovery that takes place – but it’s not the end all, be all. You’re going to really want to do something to improve business performance, either temporarily or long term. The results of the survey highlight that turning insights to action is the primary challenge that companies were facing. Just over 20 percent [of respondents] indicated that this was their biggest challenge in analytics today. There was another segment, somewhat related, that reported building trust in insights [as their biggest challenge] – and that was just under 14 percent overall. That’s 34 to 35 percent of people saying, “It’s not about me finding something, it’s about me doing something [with the data] after the fact.” That’s a pretty healthy part of the market saying that they’re having that kind of challenge.
What could be causing companies to struggle to act on the droves of data they are collecting?
Joseph: We asked companies what trend would have the most impact on the analytics profession this year, and the most common answer was the shortage of analytic professionals. I interpret that to mean that those organizations waiting and looking [to engage in data analysis] do feel pressured to find [candidates] with those skills.
This makes analytics an area of real career potential.
Joseph: My whole life, we’ve been talking about building more math and science people in the U.S. We are seeing universities come together and consulting firms sponsor education programs that focus on building data scientists.
What separates a data scientist from the rest of the analytics pack?
Joseph: The difference is that the data scientist tended to be much more in a research position. Building trust in their insights was actually their number-one challenge. Turning insights into action was number two.
They can’t do that themselves. That’s why we refer to the “analytic supply chain”: someone is working on the data, getting it formed in some way, putting it somewhere, and then someone like a data scientist is taking it out and interpreting it to look for insights. But they don’t do anything with it, necessarily. They rely on other people, like a marketing executive, to actually eye into it and then take action on it.
Without that supply chain, that connectivity to colleagues and operational staff, the data pro might fall distant from the business.
Joseph: We saw that as being less of an issue for the analyst, but still an issue. They are a little closer to the business operation. Turning insights into action was still their number-one challenge, but number two was manipulating and integrating the data. They’re having a bit more of a challenge just working with the data. So 32 to 33 percent mentioned turning insights to action was an issue versus 50 percent for the data scientists.
Your report identifies the data “science project,” the tendency of some enterprises to mine data in research and development contexts.
Joseph: [The research] isn’t tied to production. It isn’t tied to some sort of business improvement. Not necessarily that the effort includes data scientists, but that [the analysis] is theoretical and for research purposes.
What we did find is that when we looked at things like big data, there was a big difference between those data scientists and the business analysts in their grasp of big data projects. We concluded that [data scientists] were much more informed in what was taking place around big data within their organizations.
When we asked the analysts what big data tools [their company] was currently using, 73 to 74 percent were saying that they weren’t using anything or had no idea of what was being used. But when we asked the data scientists, 34 percent said they weren’t using big data tools, but only 4.8 percent said they didn’t know [what resources were used]. A lot of the data scientists were involved with Hadoop and things like that, whereas business analysts weren’t in the picture at all.
Almost half of your respondents reported that they are increasing their data quality efforts. Your survey spotlights a glaring reality of analytics: Big data doesn’t necessarily mean good data.
Joseph: We saw many more people highlighting that in their work. When there is more interest in bringing in other data from outside the company, or from data sources that are less well understood, the data quality issue goes hand-in-hand with that. Data quality can benefit from tools that are easier to use. The more we can push [good data] to the front lines, the better.
Joshua Whitney Allen is the Editor of Insights Magazine, which provides comprehensive coverage of IBM solutions for analytics, mobile, social business, and cloud, plus solution-specific content on IBM software.