NuoDB
Last Updated:
Analyst Coverage: Philip Howard
The database veterans behind NuoDB first started to explore its concepts formally in 2008 and the company was founded, at that time named NimbusDB, in 2010. The company changed its name to NuoDB the following year. The company is backed by venture capital and has its headquarters in Cambridge, Massachusetts. The company also has offices in the United Kingdom, Ireland and Bulgaria.
NuoDB’s customers are mostly in the Financial Services vertical; it generally targets use cases around cloud and hybrid cloud deployments.
NuoDB
Last Updated: 13th January 2015
NuoDB distinguishes itself from traditional relational products, not only because of its smaller footprint and reduced administrative requirements but also because it offers a (geographically) distributed database environment that can easily scale out, as opposed to scaling up. True distributed architectures have historically been difficult to implement using relational technology yet, as corporations move more and more towards hybrid cloud environments, they become ever more necessary. Which is why NuoDB is well placed.
With regard to competition from other NewSQL databases, NuoDB would argue in favour of its performance and also the fact that it has recently introduced HTAP (hybrid transaction analytic processing) capabilities, which means that you can perform operational analytics against transactional data without impeding the performance of those transactions.
Proof points quoted by the company include 1.82 million transactions per second running across 32 machines and a 72,000 database instance implementation on a single HP Moonshot system.
NuoDB leverages both direct marketing and partner models. Notable OEM and technology partners include Dassault Systèmes, Palo Alto Networks, Pentaho (for both BI and ETL), New Relic, Zabbix (for application performance management), HP, Amazon (there is a quick start kit for scale-out provisioning on AWS) and Drupal.
While the company is not focused on any particular industry sector it is particularly targeting users that are either migrating to the cloud or are deploying hybrid cloud environments. The product includes major relational database compatibility features for Oracle, SQL Server and MySQL to enable migrations from these environments to NuoDB. In the latest release there is also a new SQL-based parallel loading facility to improve performance for migrations from third party RDBMS environments.
The company’s web site lists a dozen or so customers and, although some of these are major organisations, none of them are household names. On the other hand we do know of large well-known companies that are using NuoDB in production (NuoDB worked extensively with several of them during its beta phase and is continuing to do so) but have declined to make their names public. The company is heavily targeting ISVs and application providers, especially in the financial and telecommunications markets.
NuoDB is implemented on a three-tier architecture: an administrative tier that consists of 'brokers' that provide functions such as load balancing and to which applications and users connect; a processing layer consisting of 'transaction engines' that run in memory; and a storage layer consisting of 'storage managers'. Each storage manager within a 'domain' has its own copy of the database thus, in effect, you have multiple physical instances of the same database but these are presented to the users (the transaction engines) as a single, logical instance.
NuoDB is an append-only database, which means that you cannot update a record but create a new version of that record, although actually you only write changes to the record with the most recent changes defining the current version of that record. Older versions can be reconstructed by iterating backwards through the deltas, which means that you can do point-in-time queries and have support for (logical) schema evolution. So NuoDB is heavily reliant on versioning and uses MVCC (multi-version concurrency control). In general, NuoDB uses optimistic locking as well as other techniques, not least because traditional distributed lock management tends to become a bottleneck. In cases where locks need to be applied, if there is a conflict between a read and a write then the conflict is resolved based on the time at which that request was made with the reader and writer respectively seeing the version of the data that was live at that point in time. For each process a particular transaction engine takes over the role of 'chairman' and this communicates with the other transaction engines via asynchronous messages: chairman are used to resolve conflicts in more complex scenarios and are also critical for the continuous availability that NuoDB offers.
As far as hybrid transaction/analytical processing (HTAP) is concerned, you designate a number of your transaction engines as specific to analytics. In effect, a sub-cluster is being used for transaction processing and another sub-cluster for analytics. As far as the data being processed is concerned, there is no conflict between the data being used for analytics as opposed to that used to support transaction processing, because of NuoDB’s nature as an append-only database. This means that analytics can run against the latest version of the data while not impeding updates or inserts that may be occurring within the transactional environment.
NuoDB provides the usual sorts of services one might expect: training, support, professional services and so forth. There is a free to download version of the product available, which has all the features of the main product except that it does not support geographical distribution, and is limited with respect to the number of machine hosts.
NuoDB
Last Updated: 28th June 2019
NuoDB looks relational and leverages SQL but isn’t relational under the covers. It was developed initially to support OLTP (online transaction processing) but by version 2.1 (2014) the company had started to offer a hybrid environment that not only supports transaction processing but also provides analytic capabilities. It should be noted that the product was designed like this from the outset. Under the hood, the product uses a distributed object model and data may be stored in flat file systems or block storage systems.
NuoDB particularly distinguishes itself because it offers a distributed database environment that can easily scale out, as opposed to scaling up. From a technical perspective, it may be deployed on-premises or in-cloud and it supports both hybrid and inter-cloud deployments.
Customer Quotes
“This investment and partnership demonstrates our strong belief in NuoDB’s strategy and technologies for next generation cloud-based technologies. Our partnership with NuoDB is strategic and strengthens our commitment to ensuring that our banking clients running Temenos software benefit from scale-out simplicity, elasticity and continuous availability, and that they can ultimately maximize the benefits of their shift to the cloud.”
Temenos
There are three major elements within a NuoDB deployment: an administrative tier, a processing tier and a storage tier. The first consists of “brokers” that provide functions such as load balancing and to which applications and users connect (via JDBC typically). There is also a NuoDB Insights, which provides monitoring capabilities. A major feature of NuoDB is the use of auto-administration and the ability to integrate with rules-based orchestration systems such as Kubernetes. You can define rules that automatically fire up additional resources under relevant conditions.
Fig 01 NuoDB transaction engines
As far as the processing and storage layers are concerned, these are illustrated in Figure 1. The former consists of transaction engines (TEs) that run in-memory (data is loaded on demand) and can be thought of as providing a loosely coupled distributed cache. Similarly, the storage layer consists of storage managers (SMs) where each storage manager within a “domain” has its own copy of the database so that, in effect, you have multiple physical active copies of the same database, but these are presented to the users (the transaction engines) as a single, logical instance. That said, NuoDB does support the option to partition data across storage managers but the default setting is for them to be replicas. Notably, you can have as many TEs and SMs as you like, and they can be scaled independently of one another. If required, nodes can be dedicated to analytic workloads.
Everything (data, metadata, indexes, schema and so forth) in NuoDB is known as an “atom” (sort of like an object but without object oriented properties such as polymorphism and inheritance) and transaction engines know where all the atoms are, whether in-memory or on disk, and when a transaction engine needs a particular atom it will retrieve it from the optimal source (its own memory, the memory of another engine or from disk). When changes or appends are made, that change is propagated to all other transaction engines using the same atom and is subsequently replicated to all the storage managers. Note, further, that NuoDB has a master-less architecture rather than either a master-slave or multi-master architecture, and this enables continued access to the database even in the event of multiple points of failure.
Fig 02 A typical NuoDB environment
From a hybrid processing perspective, Figure 2 represents a typical NuoDB environment with hardware isolation; meaning that operational and analytical workloads do not interfere with one another. Moreover, the implementation of MVCC (multi-version concurrency control) means that lock contention is avoided and that ACID guarantees are maintained. However, the architecture illustrated in Figure 2 supports environments where operations and analytics are distinct. There is also a class of hybrid environments where analytics, particularly machine learning or data science algorithms, needs to be embedded within operational applications. The company has told us that, in theory, NuoDB could support the import of relevant machine learning models but it has no examples of that use case within its customer base, and it does not have any specific support for third-party model libraries. On the other hand, it does support the ingestion of streaming data as there is a Kafka connector.
NuoDB primarily targets companies that want to migrate existing applications to what the company calls “Distributed SQL” environments. That is, elastically scalable, distributed, in-(multi)-cloud, container-based, SQL processing and applications. Hybrid analytic and transactional processing is a subset of that market that its direct competitors do not serve well, if at all.
The Bottom Line
NuoDB does not target what is sometimes known as “in-process” hybrid processing, but it is well suited to environments where real-time analytics is required in parallel to operational requirements.
Research
Hybrid real-time data processing
