Oracle shipped its first relational database management system (RDBMS) in 1979. A lot has changed since then, both in terms of technology and in the way data is produced and consumed. And, although the traditional RDBMS is still widely used today, the RDBMS space continues to evolve.
The traditional RDBMS is a core part of nearly every organization’s IT architecture, and for good reason. It passes the ACID test. ACID is about data guarantees, specifically guarantees of Atomicity, Consistency, Isolation, and Durability, and ACID enforces a clear contract for access to shared data. It is because of these guarantees that organizations store their most valuable data in ACID-compliant databases.
However, as technology progressed and business needs grew, the traditional RDBMS began to fall short of changing demands. The volume and velocity of data being produced, stored, and used daily has increased exponentially, and the requirement to manage new types of data through the explosion of social media, including images and text, pushed the traditional RDBMS to its limits.
Scaling to accommodate growing data needs can be difficult with the traditional RDBMS. It requires acquiring, installing, and configuring additional hardware, while taking care to protect the ACID rules. In addition, performance may suffer as complex analytics are incorporated into transactional processing.
As it always does, technology once again evolved to meet these new needs. In 2006, Amazon launched its cloud-based services. Cloud-based services have allowed organizations to scale their data applications on demand without the costly and lengthy capex projects once needed. It also relieves the burden on IT to manage the technology, allowing them to focus on the needs of the users instead. However, as users continued to demand faster and more powerful processing, it became apparent that a new database technology was needed.
At about the same time, in-memory database management systems (IMDBMS) emerged as a viable option for organizations looking to improve database performance. An in-memory database does not have the overhead of trying to optimize disk storage for reading and writing. Instead, it holds the entire database in memory all the time. Reading and writing data in memory is much faster than to data stored on a disk. But, there are disadvantages to in-memory databases, including that they may not fully support ACID, specifically durability. For in-memory databases that don’t specifically address durability, it is possible for the database to experience data loss in the event of a hardware failure because all the data was in memory. Additionally, in-memory databases require that the available memory (DRAM) be larger than the database, perhaps much larger if the database is growing rapidly. This can be expensive, or impossible.
In 2009, there were enough users of a new database structure called NoSQL to support a user conference. Sometimes referred to as “not only SQL,” this new database technology was created to manage semi-structured and unstructured data in a schema-less environment. In addition to being a valuable way to manage unstructured data, NoSQL also offered the ability to add capacity to databases by adding machines. Typically the only way to expand capacity in a traditional RDBMS such as Oracle or SQL Server was to upgrade the server it runs on. Over time, many applications that typically would have used a relational database began turning to NoSQL databases to support elastic scalability, distributed users, and emerging cloud requirements.
While the NoSQL approach provided many advantages – especially in a developer-centric environment – it soon became apparent that these databases were best for dealing with large volumes of non-critical data that don’t need the rigor of real-time transactional consistency. They were not a good fit for valuable data or scenarios in which lost or inaccurate data could be detrimental to the organization. For example, transactions regarding stock exchange fluctuations make too much of an impact on an organization’s bottom line to be swept under the rug. The focus on delivering a developer-friendly database also meant that NoSQL databases often failed to provide enterprise-level administration capabilities, making them a nightmare to actually operate in the real world.
The dichotomy resulted in a gap in the industry for applications that need the rigor and reliability of a traditional relational database, but also want the agility, elasticity, and global accessibility of that the cloud and, now, container environments can provide.
Enter the next-generation in database management systems.
Today, this emerging group of databases – what Gartner called the “avant-garde” in a 2015 report – understands the value of bringing together the advantages of cloud-based computing, in-memory, scalability, and high-performance in one system to meet the needs of a growing, global – and increasingly mobile – customer base. These systems can maintain transactional consistency and integrity at a global scale while simultaneously maintaining enterprise management capabilities. That means operating a single, logical database across multiple geographies in an active-active configuration. No sharding. No unnecessary application complexity. No loss of data.
The end result is that organizations don’t have to choose between durability and performance or between consistency and elasticity. Some of these next-generation database management systems easily scale to support data across geographies while still providing the familiarity of SQL and the assurance of ACID.
That’s not to say that an avant-garde RDBMS is the right solution for all database problems. But if you need a flexible architecture with multi-model capabilities and the ability to run in a cloud environment, considering a new RDBMS approach may complement or augment existing database solutions.
Barry Morris is an accomplished software executive with more than 25 years of industry experience in the U.S. and Europe – startups to publicly held companies. As the founding CEO of NuoDB, Barry built the company from ground up into a successful database company and a leader in scale-out SQL technology. Prior to founding NuoDB, Barry was the CEO at IONA Technologies, where he led Ireland’s most successful software company through its strongest period of growth, with more than $180 million in revenues. Born in South Africa, Barry holds a bachelor’s degree in engineering from New College Oxford University, and an honorary doctorate in Business Administration (DBA) from the IMCA. He also serves on boards of several startups in Boston, Ireland, and South Africa.
Subscribe to Data Informed for the latest information and news on big data and analytics for the enterprise, plus get instant access to more than 20 eBooks.