The Big Data Challenge: Getting from Data to Decisions in the Era of IoT

by   |   October 4, 2016 8:00 am   |   0 Comments

The Internet of Things (IoT) already has enabled connectivity in billions of devices – from thermostats to cars to wearables. But there is a new stumbling block on the horizon. Sensors are now spreading across almost every industry, triggering a massive onslaught of new data that will clearly lead us into the next era of the information age.

This reality presents both an opportunity and a challenge. On the upside, many believe that big data will unleash new opportunities for businesses, support decision-making, and lead to the development of new products and services. The question is how to get from data to decisions on a massive scale. After all, the value in big data lies in our ability to analyze and make sense of the information, and as the IoT expands many fear big data will simply become too big, too fast, or too hard for existing tools to process, analyze, and convert into insights.

Following are three key considerations to help organizations address this challenge and compete successfully in our new IoT world.

1. Capture the Right Data

As the volume of information continues to skyrocket, the variety and velocity of data will grow as well. The first and most fundamental decision facing organizations is what data to collect and keep.

Data science is all about having the data you need, which is often different from the data you have. In fact, one of the most common mistakes organizations make is failing to capture the right data needed to make the right decisions. For example, consumers are more likely to provide feedback aggressively when they are unhappy rather than when they are satisfied. Thus, for a consumer-facing company it may be misleading to make any drastic changes based on these reviews without understanding the overall context.

The question isn’t “are you collecting data,” it’s “are you collecting the right data and driving insights from that?

2. Ask the Right Questions

Knowing which data is needed usually becomes apparent by asking the right questions. Too many of today’s organizations are tempted to begin analysis before determining what problems they are looking to solve. Data should always have an earmarked purpose to prevent over-collection. Data scientists should first consider what insights they want or need to create before determining what data should be collected to execute those strategies.

3. Build Systems and Skills for the Future

Beyond creating more data, the IoT introduces a new level of statistical capabilities, data mining, and predictive analytics. Organizations will be required to transform into efficient “Data Science Machines,” and build data-processing systems that can compute at scale and in real time with high-level components.

With more data being collected than ever before, extracting value from this data is only going to become more intricate and demanding as time goes on. Decision-making systems that use statistical and machine-learning approaches to extract information from data will become critical.

But it is not just about the data and technology, there’s something else at play as well. While much of the analysis will be algorithmic and automated, in most cases humans will still be needed to connect the dots and find common threads. And for that reason, it is extremely important for organizations to make professional training a priority.

The intersection of IoT and big data is a multi-disciplinary field. In order to extract maximum value from the information available we will need teams of professionals who are trained to use the innovative tools now available to help manage big data and extract information from it. Having access to education is the foundation that effective data analysis and intelligent decision-making is built on. Skilled data scientists, statisticians and business analysts can unlock the endless possibilities of big data. Only then will we effectively address what many agree is one of the defining challenges of our times.

Devavrat Shah, co-director of the Data Science: Data to Insights course, is a professor in MIT’s Department of Electrical Engineering and Computer Science, director of the SDSC, and a core faculty member at the IDSS. He is also a member of MIT’s Laboratory for Information and Decision Systems (LIDS) and the Operations Research Center (ORC).


Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise, plus get instant access to more than 20 eBooks.

Tags: , , , ,

Post a Comment

Your email is never published nor shared. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>