A survey of Hadoop users released today found that adoption of the popular open-source storage and processing platform is accelerating.
Among the study’s key findings is the growing adoption of Hadoop: More than three quarters of the organizations that already are using Hadoop said they expect to be doing more with it within the next three months. Only 3 percent reported that they will be doing less with it over the next 12 months. Among the companies that have not yet deployed Hadoop, 8 percent plan to do so within six months, 7 percent plan to do so within seven to 12 months, and 15 percent plan to do so within a year or more.
Survey respondents represent a broad cross-section of industries, with the highest representations coming from the consulting (18 percent), financial services (14 percent), and telecommunications (10 percent) industries.
Of those respondents, 24 percent ranked their organization’s maturity with Hadoop as high, 46 percent ranked it as medium, and 24 percent ranked it as low. Online companies rank their Hadoop maturity highest, with 37 percent ranking their maturity as high. These organizations are followed by companies in the manufacturing, consulting, telecommunications, financial services, and healthcare industries, respectively.
More than 80 percent have been using Hadoop for more than six months, and 77 percent are running between 10 and more than 500 nodes – 30 percent have more than 50 nodes in place. More than 60 percent said they believe Hadoop is either “strategic” or “game changing.”
The study also found that, among companies that use Hadoop, Tableau is the most widely used BI tool. Among companies that are planning to deploy Hadoop, Excel is the leading BI tool.
Given the open-source nature of Hadoop, one might assume that cost is a primary factor driving adoption. However, only 17 percent of adopters cited cost as a primary reason for deploying Hadoop. Many more (37 percent) cited their scale-out needs as a primary consideration. Other factors identified by respondents include new applications (19 percent), revenue generation (14 percent) and curiosity (8 percent). The study also found that companies that deploy Hadoop to drive revenue and address their scale-out needs are more likely to achieve value than those looking to deploy new application or cut costs.
The study found a strong positive correlation between an organization’s maturity level with Hadoop and the tangible value it has realized. Of the organizations that ranked their maturity level with Hadoop as high, 75 percent have achieved tangible value from their Hadoop deployment, compared with 23 percent of organizations that ranked their level of Hadoop maturity as low.
Self-service access to Hadoop for line-of-business users also was highly correlated with business value. Organizations that provide business users with self-service access to Hadoop are about 50 percent more likely to realize tangible value out of their Hadoop implementation, the study found. There also is a strong correlation between self-service access and maturity level: Only 28 percent of the organizations that have been using Hadoop for less than six months provide self-service access, compared with 49 percent of organizations that have been using Hadoop for more than a year.
Another factor affecting value is executive mandate. Companies that have an executive mandate for Hadoop adoption are more likely to realize value from it: 57 percent of the organizations in which an executive mandate is driving Hadoop adoption have achieved tangible value, compared with 49 percent of organizations in which business is the main driver. In organizations with IT as the main driver of Hadoop adoption, 47 percent have achieved value.
Among Hadoop adopters, 74 percent use it for ETL, 62 percent for data science, and 65 percent for business intelligence. Among organizations planning to deploy Hadoop, however, the breakdown of intended uses changes: 69 percent intend to use it for business intelligence, 56 percent for data science, and 51 percent for ETL. The study’s authors theorize that this may indicate that using Hadoop for ETL is a transitional phase for organizations, with the ultimate intentions being to use it for data science and business intelligence.
The 2015 Hadoop Maturity Survey is published by AtScale, which partnered with Cloudera, Tableau, and MapR on the study. The study is informed by feedback from more than 2,100 respondents, making it the industry’s largest study on Hadoop maturity.
To assess your organization’s Hadoop maturity level, click here.
Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise.