How to Optimize Analytics for Growing Data Stores

by   |   December 29, 2015 5:30 am   |   0 Comments

Chad Jones, Chief Strategy Officer, Deep Information Sciences

Chad Jones, Chief Strategy Officer, Deep Information Sciences

Every minute of every day, mind-blowing amounts of data are generated. Twitter users send 347,222 tweets, YouTube users upload 300 hours of video, and Google receives more than four million search queries. And in a single hour, Walmart processes more than a million customer transactions.

With the Internet of Things accelerating at lightning speed – to the tune of 6.4 billion connected devices in 2016 (up 30 percent from 2015) – this already staggering amount of data is about to explode. By 2020, IDC estimates there will be 40 zettabytes of data. That’s 5,200 GB for every person on the planet.

This data is a gold mine for businesses. Or, at least, it can be. On its own, data has zero value. To turn it into a valuable asset, one that delivers the actionable intelligence needed to transform business, you need to know how to apply analytics to that treasure trove.

To set yourself up for success, start out by answering these questions:

What Is the Size, Volume, Type and Velocity of your Data?

The answers to this will help you determine the best kind of database to store your data and fuel your analysis. For instance, some databases handle structured data, and others are focused on semi-structured or unstructured data. Some are better with high-velocity and high-volume data.

 

RDMSAdaptiveNoSQLSpecialtyIn-MemoryNewSQLDistributed
ExampleDB2, Oracle, MySQLDeep Information SciencesCloudera, MonoDB, CassandraGraphing, Column Store, time-seriesMemSQL, VoltDBNuoDBHadoop
Data TypeStructuredStructuredUn/semi-structuredMultipleStructuredStructuredStructured
QualitiesRich features, ACID compliant, scale issuesFast read/ write, strong scale, ACID, flexibleFast ingest, not ACID compliantGood reading, no writing, ETL delaysFast speed, less scale, ETL delays for analyticsGood scale and replication, high overheadDistributed, document-based database, slow batch-based queries

 

Which Analytics Use Cases will You Be Supporting?

The type of use cases will drive the business intelligence capabilities you’ll require (Figure 1).

Analyst-driven BI.  Operator seeking insights across a range of business data to find cross-group efficiencies, profit leakage, cost challenges, etc.

Workgroup-driven BI.  Small teams focused on a sub-section of the overall strategy and reporting on KPIs for specific tasks.

Strategy-driven BI. Insights mapped against a particular strategy with the dashboard becoming the “single source of truth” for business performance.

Process-driven BI. Business automation and workflow built as an autonomic process based on outside events.

Figure 1. BI capabilities required per use case. (Source: Gartner) Click to enlarge.

Figure 1. BI capabilities required per use case. (Source: Gartner) Click to enlarge.

 

Where Do You Want your Data and Analytics to Live?

The main choices are on-premises or in the cloud. Until recently, for many companies – particularly those concerned about security – on-prem won out. However, that’s changing significantly as cloud-based solutions have proven to be solidly secure. In fact, a recent survey found that 40 percent of big data practitioners use cloud services for analytics and that number is growing.

Related Stories

Analytics in the Cloud: Find the Best Fit [Podcast].
Read the story »

With Predictive Analytics in the Cloud, the Sky’s the Limit.
Read the story »

IBM, Xilinx Eye Performance Gains for Cloud Analytics, Data Centers.
Read the story »

Systems of Discovery: Where Cloud, Big Data and IoT Intersect.
Read the story »

The cloud is attractive for many reasons. The biggest is fast time-to-impact. With cloud-based services you can get up and running immediately. This means you can accelerate insights, actions, and business outcomes. There’s no waiting three to four months for deployment and no risk of development issues.

There’s also no need to purchase and install infrastructure. This is particularly critical for companies that don’t have the financial resources or skills to set up and maintain database and analytics environments on-premises. Without cloud, these companies would be unable to do the kind of analyses required to thrive in our on-demand economy. However, even companies that do have the resources benefit by freeing up people and budget for more strategic projects.

With data and analytics in the cloud, collaboration also becomes much easier. Your employees, partners, and customers can instantly access business intelligence and performance management.

Cloud Options

There are a number of cloud options you can employ. Here’s a quick look at them:

Infrastructure as a Service (IaaS) for generalized compute, network, and storage clusters. IaaS is great for flexibility and scale, and will support any software. You will be required to install and manage the software.

Database as a Service (DBaaS), where multi-tenant or dedicated database instances are hosted by the service provider. DBaaS also is great for flexibility and scale, and it offloads backups and data management to the provider. Your data is locked into the provider’s database solution.

Analytics as a Service (AaaS) provides complex analytics engines that are ready for use and scale as needed, with pre-canned reports.

Platform as a Service (PaaS) is similar to DBaaS in that it scales easily and that application backups and data management are handled by the provider. Data solutions themselves are often add-ons.

Software as a Service (SaaS) is when back office software is abstracted through a hosted application with data made available through APIs. Remote analytics are performed “over the wire” and can be limiting.

How you leverage data can make or break your business. If you decide to go the cloud route, make sure your service provider’s database and analytics applications fit your current and evolving needs. Make sure the provider has the expertise, infrastructure, and proven ability to handle data ebbs and flows in a way that’s cost-effective for you and, equally important, ensures that your performance won’t be compromised when the data tsunami hits. Your business depends on it.

Chad Jones is Chief Strategy Officer at Deep Information Sciences.


Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise.



The Definitive Guide to Retail Analytics




Tags: , , , , , , ,

Post a Comment

Your email is never published nor shared. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>