Most companies understand the importance of being able to accurately consolidate, blend, and analyze their data to better understand what is happening in their business and to help decide what to do next. But many business teams have only scratched the surface of what is possible with data and data analytics.
The reasons for this are varied. However, for most companies, it stems from the fact that, despite decades of product-development work from business intelligence (BI) vendors, setting up and running an end-to-end BI process is complex. While a new breed of agile, cloud-based tools on the market offer easier to use, slick-looking visualization tools, the truth is that taking raw data from multiple, disparate systems in the business, blending that data together, and turning it into something that can be visualized and reliably used as the basis for everyday business decisions is not a simple task. That task is made more difficult by the increasing volume and variety of data that is generated today from different business teams using a huge number of different apps to collect data on a seemingly endless number of new business touch points.
The hardest part of the BI process is undoubtedly data preparation – the initial connection, blending, and modeling of data. The complexity of preparing data for analysis is a challenge that has plagued the BI industry since its inception. While some vendors are trying to make data-preparation tools that are simpler to use, the truth remains that consolidating and relating data from different places requires a lot of math. To innovate in this space, BI vendors have to find new ways to separate the complexity of the underlying math from the user experience. Want to learn some Lambda calculus? Me neither.
Default to Excel
Another uncomfortable truth about the BI industry is that most data analysis in businesses today still happens in spreadsheets. We see it time and time again when we start talking to companies about their existing processes for business reporting, diagnosis, and data discovery: Even though many businesses have a plethora of BI tools in place, spreadsheets are still rife.
This happens because BI initiatives are typically run by a centralized IT team with technical users. The standard corporate BI tools are purchased, controlled, and operated by IT, which leaves other business teams unable to access their data directly. As a result, business teams must endure long waits to obtain centrally generated reports from IT, with no room for ad-hoc data discovery or experimentation.
Eventually, each individual business team tries to circumvent the IT team by purchasing its own departmental analytics tools, which are typically cloud-based and easier to use. The challenge remains, however, that data preparation is inherently complex and, even with these newer tools, many business teams still struggle to get accurate, timely business answers from data. Business teams often end up continuing to rely on IT teams to curate data sources or purchasing more than one BI tool to help them build an end-to-end analytics solution that they can drive and access themselves (albeit one that breaks up the analytics workflow across different applications). All too often, business teams simply resort back to painstakingly doing their data preparation in a spreadsheet and plugging it into a visualization tool at the end of the process.
Many of the current analytics and data-preparation tools on the market also have hidden costs that aren’t exposed until the business (or team) has gone through a lengthy procurement process and chosen a new system. These hidden costs come in the form of add-ons, such as data connectors, and professional services that become necessary to get the analytics solution up and running. It’s fairly easy to compare pricing between BI vendors for software licenses, but harder to calculate the real cost of getting most BI solutions implemented. For every dollar you spend on BI software, it typically takes another three dollars to get it working. This is because the tools are usually too complex for end users to build a solution themselves and there is a huge amount of training involved. In addition, as the solution is slowly implemented, the needs of the business change, resulting in yet more professional services.
The Way Forward
I’m painting a bleak picture here, but I do believe in the unparalleled power of BI to transform businesses. Having all of your data consolidated in one place where you can quickly slice and dice it, track trends, find patterns, test correlations, and diagnose problems to get a true understanding of how your business works today, as well as how it could work better in future, is the ultimate goal. Analytics should be at the center of everything you do. Sadly, many companies do it badly, or in a very piecemeal way through spreadsheets and native analytics in individual, siloed applications.
So if I haven’t put you off the idea of BI software completely at this point, here are a few key things to think about when choosing and implementing a BI tool.
- Know your team. No matter what tool you choose, you will need to have someone on your team who understands the company’s business objectives as well as its data in order to choose the right tool initially and to do your data preparation later. Make sure that person is involved in evaluating the BI tool that you plan to use.
- Don’t make assumptions based on price alone. BI project costs can vary wildly depending on what you are looking to achieve and the level of in-house data analysis skills you have. Some companies are more upfront about total costs than others, so take published pricing with a pinch of salt at the outset.
- Look beyond visualizations. Beautiful-looking dashboards are the end product of your data preparation. Yes, bubble charts look impressive, but if the data is wrong or you can’t understand the results, they are useless. If it takes weeks to build each chart or you can’t easily slice and dice the underlying data, you’ll end up back in the world of spreadsheets in no time. It’s really easy to be seduced by a slick set of dashboards in a demo, but the important question is how they got there.
- Choose a single end-to-end BI platform. It is far better to purchase a flexible, general-purpose platform with a single, cohesive user experience than to try to use point solutions for each individual area of your business or to piece together an end-to-end solution through a combination of different BI tools. Breaking up the analytics workflow across multiple applications will reduce your agility and increase your costs. For example, if your data modeling is done in one tool and your reports and dashboard are created in another, you introduce delays into the process every time you want to make a change to the data model and see those reflected in your reports. Fewer applications is generally better.
- Start small. Choose tools that are flexible enough for you to start your analytics solution small and grow it incrementally. You don’t want to have to rebuild your data model from scratch every time you add in a new data source. You also don’t want to try to answer every possible future business question via one all-encompassing BI project. It simply won’t work, and you’ll spend forever trying to make it happen. Look for the quick wins. If it’s going to take three months or more to get your solution up and running, then either the tools you have chosen are too complex or your project is too big. Often, it’s both.
Mark Cunningham is Founder and CEO of Stytch, a data analytics company based in Vancouver, BC. Mark is a thought leader and entrepreneur with more than 25 years of experience in the business intelligence (BI) industry. He has been on the founding team of four successful analytics startups, including Indicee and Crystal Services.
Mark’s passion for solving data problems with technology began in 1991, when his family business began building the world’s first Windows-based reporting tool, Crystal Reports. Crystal grew to employ over 2,000 employees and eventually was acquired by Business Objects for $820 million. Mark later founded cloud BI company Indicee, which was acquired by Dun & Bradstreet in 2014 to form the Dun & Bradstreet Cloud Innovation Center.
Mark launched Stytch in 2016 as the only analytics platform on the market that connects to the world’s largest commercial database from Dun & Bradstreet. Stytch provides business teams with everything they need to get more insights, faster – from data preparation to dashboards.
Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise, plus get instant access to more than 20 eBooks.