A number of changes in the contemporary data landscape have affected the implementation of data governance. The normalization of big data has resulted in a situation in which such deployments are so common that they are merely considered a standard part of data management. The confluence of technologies largely predicated on big data – cloud, mobile and social – are gaining similar prominence, transforming the expectations of not only customers but business consumers of data.
Consequently, the demands for big data governance are greater than ever, as organizations attempt to implement policies to reflect their corporate values and sate customer needs in a world in which increased regulatory consequences and security breaches are not aberrations.
The most pressing developments for big data governance in 2016 include three dominant themes. Organizations need to enforce it outside the corporate firewalls via the cloud, democratize the level of data stewardship requisite for the burgeoning self-service movement, and provide metadata and semantic consistency to negate the impact of silos while promoting sharing of data across the enterprise.
These objectives are best achieved with a degree of foresight and stringency that provides a renewed emphasis on modeling in its myriad forms. According to TopQuadrant co-founder, executive VP, and director of TopBraid Technologies Ralph Hodgson, “What you find is the meaning of data governance is shifting. I sometimes get criticized for saying this, but it’s shifting toward a sense of modeling the enterprise.”
In the Cloud
Perhaps the single most formidable challenge facing big data governance is accounting for the plethora of use cases involving the cloud. The cloud’s ability to meet the storage and availability demands of big data deployments, in conjunction with the analytics options available from third-party providers, make utilizing the cloud more attractive than ever. However, cloud architecture challenges data governance in a number of ways:
- Semantic modeling. Each cloud application has its own semantic model. Without dedicated governance measures on the part of an organization, integrating those different models can hinder data’s meaning and its reusability.
- Service provider models. Additionally, each cloud service provider has its own model, which may or may not be congruent with enterprise models for data. Organizations have to account for these models as well as those at the application level.
- Metadata. Applications and cloud providers also have disparate metadata standards that need to be reconciled. According to Tamr Global Head of Strategy, Operations, and Marketing Nidhi Aggarwal, “Seeing the metadata is important from a governance standpoint because you don’t want the data available to anybody. You want the metadata about the data transparent.” Vendor lock-in in the form of proprietary metadata issued by providers and their applications can be a problem too, especially because such metadata can encompass an organization’s so that it effectively belongs to the provider.
Rectifying these issues requires a substantial degree of planning prior to entering into service-level agreements. Organizations should consider both current and future integration plans and their ramifications for semantics and metadata, which is part of the basic needs assessment that accompanies any competent governance program. Business input is vital to this process. Methods for addressing these cloud-based points of inconsistency include transformation and writing code, or adopting enterprise-wide semantic models via ontologies, taxonomies, and RDF graphs. The critical element is doing so in a way that involves the provider prior to establishing service.
The Democratization of Data Stewardship
The democratization of big data is responsible for an emergence of what Gartner refers to as “citizen stewardship” in two capital ways. The popularity of data lakes and the availability of data-preparation tools with cognitive computing capabilities are empowering end users to assert more control over their data. The result is a shifting from the centralized model of data stewardship (which typically encompassed stewards from both the business and IT, the former in accordance to domains) to a decentralized one in which virtually everyone actually using data plays a role in its stewardship.
Both preparation tools and data lakes herald this movement by giving end users the opportunity to perform data integration. Machine-learning technologies inform the former and can identify which data is best integrated with others on an application or domain-wide basis. The celerity of this self-service access and integration to data necessitates that the onus of integrating data in accordance to governance policy falls on the end user. Preparation tools can augment that process by facilitating ETL and other forms of action with machine-learning algorithms, which can maintain semantic consistency.
Data lakes equipped with semantic capabilities can facilitate a number of preparation functions from initial data discovery to integration while ensuring the sort of metadata and semantic consistency for proper data governance. Regardless, “if you put data in a data lake, there still has to be some metadata associated with it,” MapR Chief Marketing Officer Jack Norris explained. “You need some sort of schema that’s defined so you can accomplish self-service.”
Metadata and Semantic Consistency
No matter what type of architecture is employed (either cloud or on-premise), consistent metadata and semantics represent the foundation of secure governance once enterprise-wide policies based on business objectives are formulated. As noted by Franz CEO Jans Aasman, “That’s usually how people define data governance: all the processes that enable you to have more consistent data.” Perhaps the most thorough means of ensuring consistency in these two aspects of governance involves leveraging a data lake or single repository enriched with semantic technologies. The visual representation of data elements on an RDF graph is accessible for end-user consumption, while semantic models based on ontological descriptions of data elements clarify their individual meanings. These models can be mapped to metadata to grant uniformity in this vital aspect of governance and provide semantic consistency on diverse sets of big data.
Alternatively, it is possible to achieve metadata consistency via processes instead of technologies. Doing so is more tenuous, yet perhaps preferable to organizations still utilizing a silo approach among different business domains. Sharing and integrating that data is possible through the means of an enterprise-wide governance council with business membership across those domains, which rigorously defines and monitors metadata attributes so that there is still a common semantic model across units. This approach might behoove less technologically savvy organizations, although the sustainment of such councils could become difficult. Still, this approach results in consistent metadata and semantic models on disparate sets of big data.
The emphasis on modeling that is reflected in all of these trends substantiates the viewpoint that effective big data governance requires strident modeling. Moreover, it is important to implement at a granular level so that data is able to be reused and maintain its meaning across different technologies, applications, business units, and personnel changes. The degree of prescience and planning required to successfully model the enterprise to ensure governance objectives are met will be at the forefront of governance concerns in 2016, whether organizations are seeking new data-management solutions or refining established ones. In this respect, governance actually is the foundation upon which data management rests. According to Cambridge Semantics president Alok Prasad, “Even if you are the CEO, you will not go against your IT department in terms of security and governance. Even if you can get a huge ROI, if the governance and security are not there, you will not adopt a solution.”
Jelani Harper has written extensively about numerous facets of data management for the past several years. His many areas of specialization include semantics, big data, and data governance.
Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise.