When you look into your crystal ball, where do you see big data headed in 2015? In 2014, companies slowly started to adopt big data but faced challenges when applying and analyzing the data to move their business forward. In 2015, the data industry will transform completely. We foresee data science becoming a mainstream career choice as the demand for data scientists rises. Also, Hadoop will be forced to transform to integrate with heterogeneous systems of open source and traditional enterprise technologies. Finally, we see companies accepting in-memory technologies to put themselves at a competitive advantage.
Data Science Sophistication Will Increase
We predict that the coming year will see data science become a mainstream career choice. Most universities already are offering programs in data science in preparation for the dramatic rise of this new profession. High demand for data scientists will snowball in 2015, with nearly every enterprise having data scientists doing more than just studying customer behavior, which was heralded so much in 2014, expanding into new areas such as data forensics to combat rising cyber threats, fraud, and risk, as well as the creation of new types of businesses based on data.
Data science adoption will be driven by the business and its requirements, which will lead to an explosion of new tools and services focused on specific industries and departmental needs.
The Gap Between Expectations and Fulfilment Will Increase
The increased sophistication of those new tools will cause the gap between expectations and reality to grow in 2015. Newly hired data scientists are often very inexperienced regarding business questions. That inexperience, combined with the immaturity of newer data technologies, will make it a challenge to bridge the gap between business departments’ operational needs and the capabilities of technical infrastructure that is becoming more and more complex.
Data science as the science of optimizing business by exploring and mining data is becoming a mega trend, but a lot of confusion exists in the market. Many big companies already have implemented complete data science labs without any clear strategy. The role of a data scientist has to evolve next year by becoming a human interface between “data business incubators” and the actual business departments that need to gain competitive advantages over their competitors. In 2015, the need for additional qualified employees in this area will accelerate, but the number of skilled people to fulfil these requirements cannot satisfy this demand. Therefore, the gap between expectations and fulfilment will open even more.
Hadoop Will Improve Data Mining Efficiency
Hadoop certainly scores points for being a very agile and cost-effective way to store data quickly. However, the fact remains that many Hadoop projects do not lead to any real business applications. In truth, Hadoop is still an immature technology, which is a bit of a mess. The challenge for companies continues to be how to quickly and effectively analyze data stored in Hadoop, and determining how to use it to produce productionalized analytics processing in real time and at massive scale.
Hadoop is much lauded by many as a panacea for all things analytics when, in fact, companies need to be fully aware of what they are getting into when they decide to go down the Hadoop track (hence the need for data scientists). In the coming year, Hadoop systems will be forced to evolve into heterogeneous systems of open source and traditional enterprise technologies to achieve the ROI that the business requires in order to justify the costs of the system. In-memory applications and data management technologies will hybridize to integrate with Hadoop, redefining Hadoop, open source, and even operating systems. This will challenge the dominance of the mega software vendors and bruise the rising cadre of cloud systems providers.
In-Memory Will Re-Shape the IT Landscape
In-memory has become an accepted technology, and CIOs/CTOs see an in-memory layer as crucial within their complex data-management ecosystems. The need for high-performance, in-memory layers on top of data silos will accelerate significantly in 2015, extending the life of these older, siloed systems before their eventual decommissioning and redeployment in the cloud. The trend of storing more data from a growing range of sources, in addition to the availability of more business applications to leverage insights from that data, will accelerate the need for specialized, dedicated appliances to provide performance, scalability, and enterprise-ready capabilities. Many businesses have held back from diving into the in-memory pool because of all the confusion around the terminology and offerings. However, the business pressure to stay competitive will ratchet up the adoption of in-memory technologies, and companies will find themselves establishing partnerships with a new guard of technology companies in order to achieve this step-change in their competitive prowess.
As we head into 2015, we can’t expect to know exactly how these predictions will turn out, but we anticipate in-memory, data science, and Hadoop quickly gaining traction and transforming the industry in the coming year.
Aaron Auld is CEO of EXASOL and is in charge of coordinating the EXASOL’s strategic direction and positioning as well as its international expansion. He has a law degree from Munich University and an MBL in international business law from the University of St. Gallen in Switzerland. He has served as General Counsel for Ocè’s global software business (acquired by Canon) and managed the legal execution of primion’s IPO on the Frankfurt Stock Exchange as well as several merger and acquisition deals before joining EXASOL in 2006. At EXASOL, Aaron was previously responsible for Operations before becoming CEO in 2013.
Mathias Golombek is Chief Technology Officer of EXASOL. He joined the board of EXASOL AG in November 2013 and is currently in charge of the company’s technology department. Golombek began his career as a software-developer for EXASOL in 2004. He first headed the data-optimization team before taking charge of research & development in 2006. He studied Information technology at the University of Würzburg, specializing in database management systems.
Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise.