SAN FRANCISCO – Big data isn’t perfect: There are privacy issues and there probably always will be technical challenges related to the effective use of big data. But make no mistake about it, the technology is a game changer.
That was the key message Viktor Mayer-Schönberger, Professor of Internet Governance and Regulation at Oxford Internet Institute, delivered Thursday in remarks at the Connect 2016 Mobile Internet conference. In addition to several examples of how big data already is making a difference in society, Mayer-Schönberger said the benefits that big data provides are forcing us to rethink how we manage and attain insights.
“For me, big data is really all about a new perspective on reality, a new look at the world we live in,” he said. “We are armed to make better decisions.”
Scouring databases and other data stores for insight is often compared to the proverbial search for a needle in a haystack, but Mayer-Schönberger said that big data turns that idea on its head.
“With big data, we don’t know what the needle is. We can let the data speak and use it to generate really intriguing questions,” he said.
As one example, he mentioned a foreign language learning app for smartphones called DuoLingo. Millions of people use the app daily and the company decided to look at some of the data it was collecting in aggregate to see if they could identify any interesting patterns. One thing they found was that Spanish people using the app to learn English did very well until they reached a particular lesson in the series. When they re-sequenced the lessons to push the troublesome one farther back, the success rate went way up.
“The result wasn’t their intent, but they benefited from what the data told them,” said Mayer-Schönberger.
In another example, researchers in Canada were trying to discover ways to head off the tendency of premature babies to catch an infection. They attached sensors to dozens of babies who were born prematurely, tracking blood rate and other vital signs, as many as 1,200 data points per second. The results revealed a pattern indicating the likelihood of infection 24 hours before the symptoms appeared.
“They don’t know why it happens, but they gained enough information to save these babies’ lives,” said Mayer-Schönberger.
Heads or Tails?
Mayer-Schönberger compared the promise of big data to the belief that half the times that you flip a coin, it will show heads up and the other half it will show tails. Actually, he said, that’s an approximation because a coin isn’t completely evenly weighted on both sides and the thrower can’t replicate the exact same toss.
To date, we often deal in approximations, he said. “But what if we could know more? What if we could understand the world as it is in all its complexity?” He argued that, traditionally, we rate the value of data based on its ability to answer specific questions or queries and then ignore the rest of the data.
“In the future, we are going to understand that the value of data is much larger. It’s like an iceberg, and we can reap a lot of value from what’s under the water line,” he said.
Mayer-Schönberger also noted that some big data projects are making advances that government and other institutions aren’t always ready to accept. For example, he mentioned a company called PriceStats, which collects billions of price points on ecommerce sites like eBay and Amazon and uses that data to help predict inflation. In 2008, it predicted the looming financial crisis and tried to warn officials in Washington, but to no avail. (Admittedly, that was eight years ago, so perhaps now there is more understanding of big data and acceptance of the kind of insights that organizations like PriceStats and others offer).
Revolutionizing Education and Medicine
Mayer-Schönberger said there is a great opportunity for big data to revolutionize fields like education and medicine by offering much more personalized services.
“Isn’t it crazy that we all get the same medicine, like aspirin, for a cold whether we are male or female or regardless of other differences? It’s because the pharmacy doesn’t have our information” and, therefore, they have to generalize.
In a similar vein, he said it was “crazy” that most children are educated the same way. “Why are we stuck in an industrial revolution, mass-production model?” he said. “It’s because we don’t have the data.”
He urged the developers in the audience to recognize that big data offers more than a way to make money or create the latest startup. “There’s an opportunity to look beyond all that,” he said, “and help humanity understand the world better and make better decisions.”
Veteran technology reporter David Needle is based in Silicon Valley, where he covers mobile, enterprise, and consumer topics.
Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise.