What do you think of when you hear the term data warehouse? The feedback I often come across in the field includes things like “money pits,” “inflexible,” and “prone to break when changes needed to be made.” I’d lump all these under the heading of unfulfilled promises. When technologies no longer fulfill their promise, the gut reaction is to say they’re “dead” and move onto the next big thing. In reality, although it’s great for technology vendors always promoting that next big thing, it’s just too radical a move for most businesses.
That begs the question “Have data warehouses fulfilled their promise?” Perhaps not fully, but the simple fact is that data warehouses provide the mechanism to ensure the right information is available for decision making at the right time. The real question is, “Are traditional, hand-coded data warehouses dead?”
With organizations striving to more precisely target useful offers to the right consumers and businesses, with the exponential growth in the volume of data available for analysis and the increasing variety of data required to fine tune decisions, the answer is clear: data warehouses are more essential now than ever. But they will evolve beyond traditional hand-coded models. Today, enterprises require a more flexible structure that is able to adapt to change as it occurs.
The Quest for Agility
For most business managers, agility is at or near the top of every wish list. No executive wants to be left behind in the wave of dramatic, often unforeseen changes that are part of a typical business day. In a sea of business uncertainty, those waves come from all directions and staying in front of them is critical to success. In particular, IT leaders are pushing themselves and their teams to develop new approaches to agile data management, in hopes of delivering the most time-relevant fuel to analytics and business processes. Time relevance is the key determinant of data value. Making business intelligence a true competitive advantage means being the best among peers at exploiting data at its peak value. A data warehouse, built correctly, can do just that.
A published study by IDG (sponsored by SAP and featured on IT World) indicates that companies surveyed are so unhappy with their current data management platform that 49 percent report plans to evaluate new data management approaches and solutions within the next 24 months. Respondents cited a need for better real-time access and analysis of data, improved ability to support mobile workers and better ways to deliver the value of big data to employees. In short, these organizations need to do a lot more with their data.
Traditional hand-coded data warehouses are not able to help, however. They don’t allow for rapid incorporation of new data types or sources. They typically are not driven by a business information model that provides context defining precisely where new data should be integrated to be immediately usable by the business. They take too much time and too many resources to build, and organizations can no longer afford to wait 12 to 18 months for a new data mart. Today’s business environment is one where agile business processes are the objective, and an agile data foundation is required to prevent process execution bottlenecks. Competitors are making smart and informed decisions with business intelligence quicker than ever and they are stealing sales and customers from companies who haven’t figured out how to leverage their data.
Organizations need data warehouses that can leverage automation to ensure implementation in weeks—not months or years. They need systems with flexibility to ensure responsiveness to changing requirements in mere hours or days. They need an agile data foundation that is built right, built fast and built to last.
Managing Data as a Shared Enterprise Asset
In today’s big data world, many organizations are relying on facts that may be supplied by hundreds, if not thousands of their own suppliers, customers, and other business partners. Having many different data sources can make it difficult to maintain the consistency and accuracy of the facts in a data warehouse.
Our surveys of data warehouse users and operators show that nearly 30 percent of organizations take more than 90 days to integrate sources, and almost 40 percent take one to three months. As data volumes, variants and velocities increase, common sense tells us those numbers don’t get better. By creating an agile data foundation, datasets can be aggregated from multiple sources and multiple data types in near real-time.
By definition, a data warehouse is the version of truth out of which an organization draws datasets that are processed, catalogued and staged from disparate sources and data types to optimize the results of the analytics engines. Quite simply, it is the cornerstone of an organization’s information foundation. But to make data warehousing work for—not against—the organization, companies need to manage data as a shared enterprise asset by supporting the business process of data management.
Keeping Bad Data Out of the Business Environment
Today’s agile data warehouse delivers the flexibility, integrity and responsiveness required for organizations to compete and survive in a dynamic business environment. Beyond enabling agile business processes, the most effective data warehouse designs target the root cause of poor business performance by keeping bad data from infecting the business environment. This means that an effective data warehouse is actually a fully governed data warehouse. Only information that has been checked for accuracy and quality, de-duplicated, and ensured to be conformant to corporate standards is put into the warehouse. From there it can be used across the “business environment”—a system of people, processes and information—to improve metrics and KPIs which may include such indicators as order cycle time or sales outstanding.
The ability to map the associations of data sources all the way up to the KPI’s they drive, through processes that drive business performance, enables organizations to manage the data entering the warehouse and target the cause of poor business performance. This approach enables organization to treat data as a shared enterprise asset and drive tangible business value.
The most successful companies are looking beyond traditional data capture and analytical strategies to focus on technologies that accelerate the tempo of data-based decision making. The hype around big data is only contributing to their sense of urgency as companies come face-to-face with their current data management inadequacies. Organizations will need to depend on their data warehouse to provide business-contextual insights in an environment that incorporates rapidly emerging data types and new sources. Agile data warehouses, built upon the tenants of flexibility and responsiveness, provide that mechanism-and adaptable capabilities-to ensure the right information is available for decision making at the right time.
Bill Hewitt serves as Kalido’s Chief Executive Officer, President and Director, responsible for the company’s business strategy and operating results. Reach him on Twitter @KalidoCEO, and read his blog posts.