Over the next two years enterprises will rely on a mix of approaches to overcome a predicted shortage of big data and analytics professionals, according to a study by IT industry association CompTIA released Sept. 12.
The preferred method for acquiring more in-house expertise in data analysis and business intelligence is expected to be training. Fifty-three percent of the 935 respondents—all U.S. IT and business executives from an even sampling of small to large organizations—said they would invest in training current employees in big data and analytics technologies through vendor courses, conferences, certifying organizations, colleges and universities or e-learning.
Just one in three respondents to CompTIA’s July online survey said they planned to hire new employees with data and analytics expertise.
Together, these two data points suggest an underwhelming response to what is being portrayed as an overwhelming demand. According to consultancy McKinsey & Co., there will be a shortage of almost 1.7 million big data/analytics workers by 2018.
This at a time when enterprises say they recognize the importance of capturing and managing structured and unstructured data to remain competitive. The CompTIA survey respondents echoed this sentiment: 65 percent of respondents agreed with the statement, “if we could harness all our data, we would be a much stronger business.”
Failure to efficiently collect and analyze big data could cost their organizations in a number of ways, CompTIA’s respondents indicated, with 50 percent citing potential lower productivity and 43 percent citing inefficient or slow decision-making.
These insights come as a separate survey of executives at Fortune 1000 and large government agencies show 85 percent of respondents either already have big data programs or are planning to, with 85 percent of those initiatives being driven from the C-suites or from a business-unit leader. This same research, by NewVantage Partners, a management and consulting firm focused on analytics, also underscores the skills gap: Only 17 percent rated their enterprise’s “ability to use data and analytics to transform their business as more than adequate or world-class.” Harvard Business Review published a blog post on the results on Sept. 12.
Businesses Still Defining Their Skills Needs
Tim Herbert, CompTIA’s vice president for research and the author of the association’s big data study, suggests that some enterprises may be inclined to look inward for big data and analytics talent because they haven’t yet fully grasped the technology.
“Many companies are still working to understand some of the innovations we typically associate with big data,” he says. “Some of them may first attempt to do it with what they have, so they may be looking at skills in the IT department, they may be looking at general financial analysis skills, and they may be looking at others throughout the organization.”
While many enterprises may indeed identify current employees with sufficient technical or analytical skills, Herbert says organizations seeking to leverage big data may have to look outside for other types of skill sets.
“There will be these positions that fall between the technical and the analytical, and it may even include an element of behavioral science,” he says. “Those types of people don’t really exist inside a lot of companies, and those may be new hires.”
Technical, analytical or behavioral, the emerging big data scientist also must have another specialty—the business. Herbert points to a recent job posting for a data scientist by Facebook, “one of the world’s largest aggregators of data.”
The ad emphasizes that the data scientist must communicate with product managers and engineering teams and produce “actionable” guidance.
“He or she must be able to positively impact the bottom line,” Herbert writes in the published study.
One of the industries in which big data and analytics specialists will be most in demand is IT. Technology vendors, Herbert says, are keenly aware of the demand among their customers for big data and analytics expertise, a demand many are rushing to fill through consultancy practices.
“A lot of the big (IT) companies are scrambling to ensure they can provide a solution to customers who inquire about big data,” he says. “There’ll be a significant amount of training and hiring within the IT companies themselves. And usually they’re ahead of the curve, so a lot of the hiring will happen with them before it happens within the customers.”