Big Data Analytics Predictions for 2016

by   |   January 4, 2016 5:30 am   |   4 Comments

Editor’s Note: For more insights from big data analytics experts looking ahead to 2016, download our eBook on predictions for 2016 here.


fireworks 2

Happy New Year!

The celebrating is over and many of us face 2016 with excited anticipation of how big data analytics will enable astonishing innovations that create new industries and reinvent others. Others look ahead with trepidation, unsure of what the New Year holds for big data.

To shed some light on the road that lies before us, experts from across the big data landscape weigh in with their views for what we can expect in see in the world of Big Data Analytics in 2016. We have organized their insights based on the category to which they apply.

Best wishes for happiness and success in 2016.

Internet of Things

Big data cloud services are the behind-the-scenes magic of the Internet of Things (IoT). Expanding cloud services will not only catch sensor data but also feed it into big data analytics and algorithms to make use of it. Highly secure IoT cloud services will also help manufacturers create new products that safely take action on the analyzed data without human intervention. – Neil Mendelson, VP, Big Data Advanced Analytics, and Jeff Pollock, VP, Big Data Integration and Governance, Oracle

The Industrial Internet knits together big data, machine learning, and machine-to-machine communications to detect patterns and adjust operations in near real time. Soon, the Industrial Internet will expand by definition to include the Internet of Things.

I predict that, in the very near future, real-time data streams will transform what is possible across the Industrial Internet so users can ask critical questions, adjust a process, or see a pattern in the moment. Entire industries such as energy, pharmaceutical, and even agriculture will be dramatically impacted by the ability to analyze real-time and historical data together to make business decisions faster. – Eric Frenkiel, CEO, MemSQL

As we move forward through 2016 and beyond, more devices, agents, sensors, and people will join the IoT. Perhaps we will even progress as a society to a post-scarcity economy, and information itself will become our commodity of trade. Monetizing the exchange of information, micro-licensing, and transactions become prominent tasks as our automation and machine-to-machine networks take care of daily needs. Imagine algorithms as apps for applying big data analysis over the connected masses of information generated by the IoT and its billions upon billions of connected devices in every aspect of our lives. Owning the data, analyzing the data, and improving and innovating become the keys to corporate success – all empowered by a connected digital society.

Though this may have some Orwellian overtones, the IoT is really about the Zen of Things – our application of software and technology to help customers consume products and to help businesses build better products and deliver better services. In 2016, the IoT will continue to combine big data, analytics, the cloud, artificial intelligence, robotics, and automation to propel industries forward and create the next industrial revolution. – Mark Barrenechea, CEO, OpenText

Machine Learning

In 2016, every company will want to get on the machine-learning bandwagon. But without the right people, many won’t have the expertise to do it. Expect to see the development of turnkey databases that allow developers to build predictive models without having a Ph.D. – Monte Zweben, co-founder and CEO of Splice Machine and executive chairman of RocketFuel

Related Stories

Data Analytics in 2016: Steady Acceptance and Widening Horizons.
Read the story »

Cloud Computing Moves to the Edge in 2016.
Read the story »

2015: A Transformative Year for Big Data.
Read the story »

Prepare for the BYOD Holiday Hangover.
Read the story »

As it gets increasingly difficult to find data scientists to mine the rapidly growing volumes of data for business insights, companies will turn to autonomous services for machine learning. These include offers such as Azure ML Studio from Microsoft, the Google Prediction API, Amazon Machine Learning, and IBM’s Watson Analytics, which all are helping to make machine learning more accessible. – Roger Levy, VP of Product, MariaDB

You will hear a lot of excitement and hype about artificial intelligence, with many startups and large companies making major investments. However, commercial success will go to firms that quietly apply more narrow machine-learning techniques to solve relatively mundane problems, such as delivering more personalized product recommendations, promotions, and services, without much fuss or fanfare. Those that can convert the big futuristic ideas into more manageable practical applications and use them across the entire organization will achieve the greatest return on their investment. – Anatoli Olkhovets, Vice President, Product Management, Opera Solutions

Big data has become so ubiquitous and integrated into everything we do – from reading a map to shopping online or in a store – that it is no longer a separate topic of discussion. In 2016, technologists will shift their attention from big data to machine learning and providing proactive insights. “Active intelligence” will become the new focus, whereby companies will leverage technology like predictive analytics and machine learning to provide solutions that are actively analyzing data 24/7 and alerting us when significant events happen. – Tim Barker, CEO, DataSift

2016 will be the year in which artificial intelligence (AI) technologies, such as machine learning (ML), natural language processing (NLP), and property graphs (PG) are applied to ordinary data processing challenges. While ML, NLP, and PG already have been accessible as API libraries in big data, the new shift will include widespread applications of these technologies in IT tools that support applications, real-time analytics, and data science. – Neil Mendelson, VP, Big Data Advanced Analytics, and Jeff Pollock, VP, Big Data Integration and Governance, Oracle

Artificial intelligence and cognitive computing will make personalized medicine a reality, help save the lives of people with rare diseases, and improve the overall state of healthcare in 2016 and beyond.

Although President Obama launched a precision medicine initiative in his 2015 State of the Union address, today only about 15 percent of hospitals use predictive analytics, and practicing personalized medicine continues to be out of reach for most medical professionals. But recent technical breakthroughs that apply artificial intelligence to big data predictive analytics make it possible to do the following:

    • Practice personalized medicine by mapping genomic data with individual patient electronic medical records.


    • Practice precision medicine by answering questions about treatments/medicine for sub-populations such as children, older adults, women, and different ethnicities.


  • Answer questions for treatments/medicine for rare diseases, which has been cost-prohibitive in the past due to small sample sets. – Dr. Jans Aasman, Cognitive Scientist and CEO of Franz Inc.


Data Security

The year 2015 was one of escalating breaches for banking, healthcare, government, media and telecommunications. No industry sector was spared, and these attacks demonstrated their destructive capabilities. Nation-state activity increased to an all-time high, paving the road for the cybersecurity pact with China. From a technology point of view, social, mobile, big data, and cloud transitioned from buzzwords to the new normal.

In 2016, I expect cyberthreats will continue to increase. Whether or not the cybersecurity pact leads to a framework of new international norms remains to be seen. Cloud continues to mature and will see adoption by large companies that only a year or two ago would have never considered it as an option.

In 2016, cloud will be about leveraging new capabilities rather than just a cost savings. Analytics and cognitive capabilities will see rapid growth as organizations look at their big data for new insights.

IoT will continue to grow as new devices are introduced regularly, and IOT device makers will be challenged by the amount of data being collected and how to properly safeguard that information. Additionally, privacy laws will continue to evolve, challenging organizations on their appropriate use of data. – David Cass, CISO, IBM Cloud and SaaS Operational Services

Click to enlarge

Click to enlarge

Increasing consumer awareness of the ways data can be collected, shared, stored – and stolen – will amplify calls for regulatory protections of personal information. Expect to see politicians, academics, and columnists grappling with boundaries and ethics. Companies will increase use of classification systems that categorize documents and data into groups with pre-defined policies for access, redaction, and masking. The continuous threat of ever more sophisticated hackers will prompt companies to both tighten security, as well as audit access and use of data. – Neil Mendelson, VP, Big Data Advanced Analytics, and Jeff Pollock, VP, Big Data Integration and Governance, Oracle

With the constant threat of breaches, organizations will embrace multiple layers of security. In addition to an increased focus on access and authorization, companies will start to implement native encryption to protect data as it resides in the database and SSL encryption to protect data as it moves between applications. – Roger Levy, VP of Product, MariaDB

It seems as though major data breaches are happening daily. From banks to retailers to government agencies, bad guys are accessing personal data at a staggering rate – millions of records at a time. In the upcoming year, I believe businesses that have been collecting big data without putting thought into what they really want to collect – what is useful, what is superfluous, what is risky to store – will start to suffer from big data indigestion. Businesses need to put greater care into governance or face serious consequences due to the cost, liability, dangers, and headaches involved in storing so much sensitive-yet-unnecessary data spread across the organization. – Scott Zoldi, Chief Analytics Officer, FICO

Data Democratization

In 2016, companies will make even greater strides at becoming insight-driven enterprises. We will see analytics insights being embedded throughout more of the business processes at the right time, and through collaborative and immersive decision-making environments that encourage dialogue and decisions that drive action. Companies also will place a big focus on extending their operating models to support this shift – this includes talent, technology, and business processes that support moving from insights to decisions to actions to outcomes. As companies develop a strong insight-driven culture and apply the data and analytics technologies available to them, whether it’s data visualization tools for decision making or machine-learning techniques embedded in automated decisions or recommendations across the enterprise, they are operationalizing a strategy that can help them to move or remain a step ahead of their competition and/or become an industry disruptor. – Brian McCarthy, Managing Director, Accenture Analytics – Analytics Advisory Lead

Companies will wake up to the harsh reality that if their data is locked away in an ivory tower, accessible only to data scientists instead of customer-facing employees, any insights to be found within will take much longer to implement, resulting in missed opportunities. In 2016, companies will put key insights or recommendations into the hands of employees at the point that a decision needs to be made, so the company can derive the most value. – Anatoli Olkhovets, Vice President of Product Management, Opera Solutions

In the coming year, simpler big data discovery tools will let business analysts shop for datasets in enterprise Hadoop clusters, reshape them into new mashup combinations, and even analyze them with exploratory machine-learning techniques. Extending this kind of exploration to a broader audience will both improve self-service access to big data and provide richer hypotheses and experiments that drive the next level of innovation. – Neil Mendelson, VP, Big Data Advanced Analytics, and Jeff Pollock, VP, Big Data Integration and Governance, Oracle

Democratization of governance will drive collaboration. Data governance is no longer just the domain of IT and compliance teams. Today, self-service and collaborative data management dictates that everyone has a shared responsibility for ensuring the quality and security of information across the enterprise. Business users will get involved with the quality and governance of data, as partners with IT, by adding value through social collaboration on data sets through the course of their day-to-day activities. – Manish Sood, founder and CEO, Reltio

Open Source

MapReduce is quite esoteric. Its slow batch nature and high level of complexity can make it unattractive for many enterprises. Spark, because of its speed, is much more natural, mathematical, and convenient for programmers. Spark will reinvigorate Hadoop and, in 2016, nine out of every 10 projects on Hadoop will be Spark-related projects. – Monte Zweben, co-founder and CEO of Splice Machine and executive chairman of RocketFuel

It’s hard to believe that Hadoop is over 10 years old. While interest remains strong and usage is maturing, there continues to be new options that either complement or provide an alternative to Hadoop to handle Big Data. The rapid ascension of Apache Spark and Apache Drill are examples. We’ll continue to see more options in the New Year. – Manish Sood, founder and CEO, Reltio

Companies will want to address multiple database types in one system versus having to juggle several different ones. We’ll definitely see a rise tools such as Cassandra for combining JSON, SQL, NoSQL, etc. – Roger Levy, VP of Product, MariaDB

From the Internet of Things to healthcare to cyber terrorism, it’s no longer just about gathering and analyzing data. It’s about gathering, analyzing, and acting on data as it happens. With hardware commoditized (or bypassed entirely in favor of the cloud), and open-source software (e.g., Apache Ignite, Spark streaming, Storm) coming into its own, it is now economically feasible to squeeze even more value out of data in real time. A cornerstone of prescriptive analytics, streaming analytics will come of age in 2016. – Scott Zoldi, Chief Analytics Officer, FICO

Data Management

Next-generation, software-based storage technology is enabling multi-temperature (fast and dense) solutions. Flash memory is a key technology that will enable new design for products in the consumer, computer, and enterprise markets. Consumer demand for flash will continue to drive down its cost, and flash deployments in big data will begin to deploy. The optimal solution will combine flash and disk to support both fast and dense configurations. In 2016, this new generation of software-based storage that enables multi-temperature solutions will proliferate so organizations will not have to choose between fast and dense – they will be able to get both. – John Schroeder, CEO and co-founder, MapR

Back in 1977, Junior Murvin sang about “police and thieves in the street.” Leaving aside the thieves for now, I think we will be seeing a lot more police on our streets in the coming year. And in this day and age, they more than likely will be wearing a body camera and capturing hours and hours of video footage, which will need to be analyzed and stored for a significant amount of time. Just in case.

The requirement will be for very large amounts of highly secure, incorruptible long-term storage. The judiciary will stand for nothing less. And of course because public money is involved, it will have to be economical. – Nik Stanbridge, VP Marketing, Arkivum

Knowing where data comes from – not just what sensor or system, but from within which nation’s borders – will make it easier for governments to enforce national data policies. Multinational corporations moving to the cloud will be caught between competing interests. Increasingly, global companies will move to hybrid cloud deployments with machines in regional data centers that act like a local wisp of a larger cloud service, honoring both the drive for cost reduction and regulatory compliance. – Neil Mendelson, VP, Big Data Advanced Analytics, and Jeff Pollock, VP, Big Data Integration and Governance, Oracle

While companies are already discovering the benefits of a hybrid cloud strategy, such as greater scalability and flexibility, better business continuity, disaster recovery, and capital cost savings, operating across multiple environments also presents challenges. Setting up this migration can be difficult and costly if not implemented properly. Expect to see solutions making this process smoother enter the marketplace. – Roger Levy, VP of Product, MariaDB

The idea of storing all aspects of information and deriving business intelligence is revolutionizing the IT economy. However, with the Big Bang of information, data complexity and scale are stifling this revolution. Salvation can only be found in the simplification and unification of data with one integrated solution for both structured and unstructured data.

In order to achieve this, the way data is stored and presented needs to fundamentally change. Instead of trying to engineer technologies to layer on top of existing database solutions, a new science is needed to bring structured and unstructured data together from the bottom up. And that means doing away with traditional data warehousing (which can’t keep up with the pace or variety of big data anyway) and storing all data in its native, raw state. A new science that makes information singularity (unification with simplification) a true possibility, and makes it possible to collect any kind of data, regardless of form or model, and get exactly what you want out of it, or view it exactly how you want, all in real-time.

Only then will all the big data insights we keep anticipating be possible. – Thomas Hazel, founder and CTO, Deep Information Sciences

NoSQL and graphs concepts are permeating the enterprise landscape, where the schema-on-read, high scalability, and real-world representation of relationships are prized. Google, for example, uses a combination of Big Table and their Knowledge Graph. In 2016, we’ll see NoSQL and graphs take a leading position in the marketplace. – Manish Sood, founder and CEO, Reltio

Businesses are no longer simply experimenting with NoSQL, they are now re-platforming their entire infrastructure around it to support their Web, mobile, and IoT applications. In 2016, we’ll see more enterprises re-platform their data management systems using NoSQL to overcome the limits of their 30-year old legacy relational systems. – Bob Wiederhold, CEO, Couchbase


Mobile applications must be fast, responsive, and reliable to meet end users’ demanding expectations. To meet these demands, mobile developers are no longer building applications for the best-case performance scenario; they are building apps for the worst-case scenario – guaranteeing an ideal mobile experience regardless if connectivity is ideal or spotty. To stay competitive, companies will take an offline-first mobile development approach so business performance is no longer tied to uncontrollable network performance. – Bob Wiederhold, CEO, Couchbase

Mobile UI will emerge as the new normal. With the ubiquity of mobile devices in every shape and form factor, and the new normal of accessing applications on the go in every facet of our day-to-day lives, mobile UI for enterprise applications will become a prerequisite, and not just a luxury, in 2016. – Manish Sood, founder and CEO, Reltio

Scott Etkin is the editor of Data Informed. Email him at Follow him on Twitter: @Scott_WIS.

Editor’s Note: For more insights from big data analytics experts looking ahead to 2016, download our eBook on predictions for 2016 here.

Subscribe to Data Informed
for the latest information and news on big data and analytics for the enterprise.

The Definitive Guide to Retail Analytics

Tags: , , , , , , , , , , , , , , , , , ,


  1. Posted January 12, 2016 at 5:09 am | Permalink

    Hello Scott, very nice and informative post!

    I would like to add something into it: Modern data integration is now a part of big data analytics, and moving other big data platforms to Hadoop will be a major part of big data integration in this year.

  2. Scott Etkin
    Posted January 18, 2016 at 12:40 pm | Permalink

    Data integration is important as it lends important context to the data you are analyzing, so I suspect you are correct.

  3. Patrick Hagan
    Posted January 20, 2016 at 11:29 am | Permalink

    Hello Scott Etkin, a very good summary of all the latest and greatest technologies. I also agree that data integration is an important part of big data analytics, but data integration has always been the holy grail. I disagree that Hadoop is a major part of big data integration. It might be, but it seems the industry from what I have read is moving towards Spark on other storage platforms and ultimately to some sort a distributed OS. May you live in interesting times. Thank you.

  4. James Church
    Posted February 3, 2016 at 1:20 am | Permalink

    Big data is a pool of information to develop insight. The written article is really appreciable effort and contains amazing elaborations about big data and its active presence online.

Post a Comment

Your email is never published nor shared. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>