Ask Clay Richardson, a Forrester analyst, about how to make the most of big data and you will hear his favorite mantra: Big data demands big process. The implication of that statement is that big data can’t just be allowed to “happen,” Richardson says. Planning and technology need to be brought to bear if you are going to wring the benefits out of all of that data. In particular, big and powerful tools are going to be mandatory.
One element of that response, according to Richardson and others, should be business process management (BPM), which can be used to streamline the data handling process and add intelligence to the management of big data. Richardson says BPM can help turn data into visualizations rapidly enough to give data immediate value to business end users. And, he notes, BPM is a business-friendly approach to handling the big data challenge that requires less hands-on help from IT.
BPM has been widely embraced in recent years for its ability to tame and organize processes. A BPM system (or BPM software) embodies this philosophy and the terms are often used to describe a way of managing business processes and connecting people, business functions, and applications.
In the vision described by BPM proponents, a marketing department intent on parsing useful information from clickstream data and company social media activities might choose to lean on a BPM system as its big data manager. The BPM system, in conjunction with analytic tools, could help plow through big data rapidly and with more automation than is often the case. A particular process of interest could be graphically represented by a business user and then quickly turned into an executable function allowing specific customer activities (or a series of activities) to be identified and reported to a sales professional—or combined with CRM data to help enhance sales force effectiveness.
Although some organizations might find a more “hard-wired” approach to big data useful, for instance where the input is not expected to vary substantially or where responses are likely to be calibrated within a known range. However, say proponents, for others looking to be more adaptive and agile, BPM can be a perfect starting point.
BPM Not an Answer by Itself
Richardson points out that BPM alone won’t solve all the problems. Enterprises employing BPM in this context will still need to address data quality and a master data management (MDM) system should be set up to work hand-in-glove with BPM.
Michele Cantara, Gartner’s vice president for BPM, envisions a similar approach to deploying a BPM system to support companies working with big data analytics. She says having big data without being able to continually put the data in context means you really have just a “bunch of stuff” rather than actionable information.
“BPM is important because it puts the data into a process context and in the context in which the work is being done. That means you can make better decisions and perform informed analysis,” she says.
Cantara says you can achieve this synthesis of big data and BPM through several means, including a BPMS platform (a Gartner term for a complete BPM suite), which can “instrument the data and put it in a process context.” But implementing BPM should be done within a broader context. For example, she notes, “You clearly need to have an enterprise architecture (EA) before you start on any kind of business process improvement, including BPM, and to do that you need to understand business considerations and strategy,” she says.
Big data is disrupting traditional information architectures — from a focus on data warehousing (data storage and compression) toward data pooling (flows, links, and information shareability), says David Newman, another Gartner analyst, in a recent press release. “In the age of big data, the task for the EA practitioner is clear: Design business outcomes that exploit big data opportunities inside and outside the organization,” Newman adds.
A Means to Simplify Big Data Management
From a practitioner’s standpoint, David T. Moore, a TIBCO service-oriented architecture (SOA) enterprise architect and partner at his own consulting firm, SOAMoxie, says the integration that BPM enables can encapsulate big data processes, such as parsing and cleansing – greatly simplifying big data management.
The bottom line, he says, is that MDM or other data processes can be streamlined using BPM, enabling organizations to achieve and capitalize on better data analysis. That, in turn, can result in improved competitiveness or concrete achievements like getting to the marketplace first, for example.
From the business perspective, Moore says BPM and MDM can be integrated, programmatically, into workflows. “Without getting into coding details, BPM can be made into an automated process. In the old days the same thing might have been achieved by having a remote procedure call but now, with messaging systems to invoke, we can plug and play and say, I’m finished with this step, now let’s do these other things,” he explains.
To be sure, acknowledges Moore, this step-wise processing is still done “the old way” in many large organizations and government entities – perhaps with a workflow in place but a human beings still manually entering data.
However, he notes, “BPM can integrate the MDM as well as the ETL (extract, transform and load) process as part of the workflow, becoming an actual process that integrates larger activities.
For those interested in using BPM with big data, Moore says the first step to take depends on the current state of what you are working on. Understanding that may take time. “Nobody has all their ducks in a row,” he says.
Finally, adds Cantara, businesses should look at how emerging BPM capabilities may further refine its use with big data. She notes that Gartner has defined intelligent BPM systems (iBPMS) as the next evolution of BPM – incorporating more analytics and other technologies into process orchestration.
These systems will allow organizations to have more intelligent processes by enhancing a businesses situational awareness and more quickly identifying patterns of interest. “In the age of social media and continual change, this will become critical,” Cantara adds.
Alan R. Earls is a business and technology writer based near Boston.