The Open Data Platform Initiative (ODPi), a nonprofit consortium of big data industry leaders focused on the simplification and standardization of the big data ecosystem, announced today the release of its ODPi runtime specification and test suite to ensure applications will work across multiple Apache Hadoop distributions.
Descending from Apache Hadoop 2.7, the runtime specification includes HDFS, YARN, and MapReduce components and is part of the common reference platform ODPi Core.
The ODPi test framework and self-certification align closely with the Apache Software Foundation by leveraging Apache Bigtop for comprehensive packaging, testing, and configuration. More than half of the code in the latest Bigtop release originated in ODPi.
The runtime specification and test suite were designed to ensuring interoperability across the Hadoop ecosystem. A lack of standards The ODPi believes that the current ecosystem is slowed by fragmented and duplicated efforts.
“We aim to speed Hadoop adoption through ecosystem interoperability rooted in open source so enterprise customers can reap the benefits of increased choice with more modern data applications and solutions,” said Alan Gates, co-founder of Hortonworks, an ODPi member organization. “We are pleased to see ODPi’s first release become available to the ecosystem and look forward to our continued involvement to accelerate the adoption of modern data applications.”
“The turbulent big data market needs more confidence, more maturity, and less friction for both technology vendors and consumers alike,” said Nik Rouda, senior big data analyst at Enterprise Strategy Group (ESG). “ESG research found that 85 percent of those responsible for current Hadoop deployments believed that ODPi would add value.”
The published specification also includes rules and guidelines on how to incorporate additional, non-breaking features, which are allowed provided that source code is made available through relevant Apache community processes.
“Big data is the key to enterprises welcoming the cognitive era, and there’s a need across the board for advancements in the Hadoop ecosystem to ensure companies can get the most out of their deployments in the most efficient ways possible,” said Rob Thomas, vice president of product development at IBM Analytics. IBM is a member of ODPi. “With the ODPi runtime specification, developers can write their application once and run it across a variety of distributions, ensuring more efficient applications that can generate the insights necessary for business change.”
“With its broader, flexible approach to standardizing the Hadoop stack, ODPi is particularly attractive to smaller companies,” said Milind Bhandarkar, founder and CEO of ODPi member organization Ampool. “Instead of spending testing/qualification cycles across different distributions and respective versions, the reference implementation would really help reduce both the effort and risk of Hadoop integration.”
Later this year, ODPi will make available its operations specification, designed to help enterprises improve installation and management of Hadoop and Hadoop-based applications. The operations specification covers Apache Ambari, the Apache Software Foundation project for provisioning, managing, and monitoring Apache Hadoop clusters.
“ODPi complements the work done in the Apache projects by filling a gap in the big data community in bringing together all members of the Hadoop ecosystem,” said ODPi senior manager John Mertic. “Our members – Hadoop distros, app vendors, solution providers, and end-users – are fully committed to leveraging Apache projects and utilizing feedback from real-world use cases to provide industry guidance on how Hadoop should be deployed, configured, and managed. We will continue to expand and contribute to innovation happening inside the Hadoop ecosystem.”
Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise.