New research by Warwick Analytics shows that the number of data scientists in the UK looks to grow by around 50 percent in 2017. However, skilled resources are scarce and a key constraint is the amount of time data scientists manually spend processing data. This means that the key to unlocking the full potential of the UK data-science market is not just to train more data scientists, but to speed up a lot of the manual processes that transform and prepare data for analysis.
The research was carried out amongst 70 leading UK data scientists working across a range of commercial sectors, including finance, retail, consumer packaged goods and manufacturing.
When looking at the respondents’ recruitment plans in 2017, the number of data scientists in the UK looks set to grow by 54 percent year-on-year for enterprises who consume their own internal analysis from their own data science teams, and 21 percent for providers of data science to other companies.
This number varied significantly amongst specific sectors however, with retail and CPG expecting the highest rise in the number of new data scientists—a staggering 125 percent. The finance industry is expecting 71 percent growth, but the utilities, media and communication industries are expecting an increase of just 12 percent.
There are positive signs, too, when it comes to the development of new analyses within companies: 87 percent of respondents expect to be spending their time building more models that are more complex than existing ones. Seventy percent will also be focusing on enhancing existing models with richer data and 50 percent will be building more models like existing ones.
However, the research also showed that data scientists in the UK spend up to 80 percent of their time having to cleanse and wrangle with data before any analysis or insight is even extracted. If this can be automated or sped up, it could potentially double the output.
The size of the daily problems was also revealed, with the data scientists interviewed typically dealing with billions of rows of data each day, both structured and unstructured. The most common type of data being analysed today was tabular structured data, with a high proportion also being text, weblogs and IoT, all of which are fast-growing. Due to restrictions and challenges in the capability to handle unstructured data, these were analysed least, and there was a near-universal aspiration to be able to more easily enrich analyses with unstructured data.
Sales and marketing, operations, finance/risk, customer service and quality were the most popular functional applications of data science, whilst customer insight was the most popular use case that data scientists were predominately solving.
Professor Mark Girolami, a Director at the Alan Turing Institute and Chair of Statistics at Imperial College, said: “The UK is a leading authority when it comes to Big Data and it’s great to see the Data Science industry growing significantly. The main challenges for companies is to find the right talent which has both commercial and academic expertise, which is scarce, and also to help them spend more time on analysis and less on data handling.”
In a booming market, there is always a concern about hype. However, we are delighted to see commercial data science truly flourishing in the UK. It seems that the discipline is solving real commercial problems on a day-to-day basis. Indeed it seems that the challenges now are to cope with the demands and possibilities to create value. Whilst we do not see that AI bots can somehow replace the skill and ‘artistry’ of the data scientist in terms of thinking through the methods to resolve issues, we do believe that there’s a lot of ‘heavy lifting’ being done by scarce, skilled resource for example in transforming and cleansing data. We believe that automation tools and platforms can help to alleviate some of these tasks as well as opening up new possibilities of analyses with heterogeneous data which are clearly desired but not being done because of resource constraints.
Dan Somers, CEO at Warwick Analytics
Dan is a serial entrepreneur and CEO of Warwick Analytics. A prevalent speaker and widely published in the predictive analytics arena, Dan has also founded other successful IT businesses, including the managed services provider VC-Net. He holds an MA from Cambridge University in Natural Sciences and a Diploma in Business Studies.
Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise.