Large companies and organizations are already using big data for some specialized applications, but the majority of that big data is still structured data (transactions, clicks, etc.). It is predicted that in 2017 we will see a greater focus on leveraging unstructured data as big data usage continues to rise.
Many digital companies whose businesses rely on big data have been leading the pack in their ability to gain beneficial insights from unstructured data. But this ability is now expanding to enterprises that aren’t as data-reliant as more data becomes available, and as the tools to manage that data become attainable without huge investments or long development cycles. This will allow big data analysis in 2017 to more frequently include insights derived from an organization’s vast stores of disparate, unstructured data in addition to the current popular focus on making sense of the numbers, analytics, and business intelligence provided by structured data.
This will be a fundamental shift. Structured data provides important numbers about revenue performance, operational metrics, etc., but unstructured text holds the critical information about how business actually gets done. A company’s institutional knowledge (or secret sauce), discoveries, internal processes, and competitive edge are often contained in a vast array of written text. It takes natural language processing, unified information access, and cognitive search capabilities to extract information from this text and share it in a useful way with those who need it. This will allow organizations to understand what the text is saying on a broad scale and use that to drive innovation, increase efficiency, and improve operational effectiveness.
The Rise of Search Technology to Tackle Persistent Challenges
Two challenges often plague organizations as they try to tackle unstructured data: infrastructure and unification. Businesses are starting to solve the infrastructure problem with data lakes or the cloud, but the unification of unstructured data remains tricky. Of the “three Vs” of big data (volume, variety, and velocity), the one that poses the greatest challenge to unstructured data usage is that of data variety. This vast amount of variety makes successful unification absolutely critical. But data lakes, while providing more flexibility than traditional data warehouses, don’t solve the variety conundrum; they just postpone the need to solve the problem to the point of data usage rather than the point of ingestion.
In 2017, the use of search technology to create a logical data warehouse will gain significant traction as a more flexible alternative for solving the data unification problem. Search technology is key for bringing together disparate data from data stores and silos across the organization – without having to build a data lake. Adoption of unified insight engines will be on the rise in 2917, which will allow companies to access and quickly gain insights from the content buried in their vast unstructured data stores. Creating a logical data warehouse with the technology of today’s insight engines takes only weeks and supports both structured and unstructured data for a faster and more comprehensive solution.
A Sample Use Case: The Healthcare Industry
Healthcare is an industry that holds tremendous promise for improved unstructured big data analytics. The human body is immensely complex with high variances, and the industry has amassed huge repositories of data that we’re just starting to fully understand. More plentiful and higher quality data holds tremendous potential for unpacking meaning alone, but the real treasure trove lies in understanding relationships and interactions within and between the documents and data. Leveraging unstructured in addition to structured data will not only help bring better drugs to market faster, but also accelerate cures, reduce healthcare costs, and improve quality of life through things like personalized medicine. These developments won’t hit the mainstream in 2017, but the pace of innovation is accelerating.
The biggest hurdle here is, again, the unification of all kinds of information regardless of source, silo, or type. Another hurdle exists in the need to develop the ability to analyze structured and semi-structured healthcare data in context, thus bringing some of the techniques of unstructured data analysis to structured data. The human genome, for example, generates structured data, but analysis on that kind of data isn’t the same as analysis on revenue figures or last quarter’s sales results. Context is crucial, and that context is made usable only with the same semantic analytics techniques that are required for extracting meaning from text, or unstructured data.
Big data use cases and analysis technologies will make tremendous strides in 2017. By embracing natural language processing, unified information access, and the advanced capabilities of today’s cognitive insight engines, organizations of all sizes and from various sectors will be able to leverage their unstructured data and better understand the context of their structured data. By doing so, they will add meaning, nuance, and sentiment to their big data analyses, leading to efficiencies, strategies, and discoveries that will change the way they do business and — in some cases — change the world.
Jeff Evernham is the Director of Consulting for North America at Sinequa, where he leads Sinequa’s expansion in North America with responsibility for client engagements, sales engineering, solution delivery, and partner management. He specializes in aligning cutting-edge technology solutions with business needs, with over twenty years of experience in software, professional services, and management consulting. Jeff has deep expertise in data analysis and business intelligence, and led the analytics and visualization practice at Knowledgent, a big data and analytics consulting firm. He was instrumental in the rapid growth of Synygy, a software and services provider, where he served for over 15 years, attaining the role of Vice President of Global Professional Services. He began his career as a Technical Specialist at The Boeing Company after graduating with Bachelor and Master of Science degrees in Aerospace Engineering from MIT.
Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise.