Amazon MemoryDB

Amazon offers a range of different database services within its AWS cloud platform. At the time of writing their website lists no less than nine different Amazon databases, including RDS (a Postgres database), Neptune (a graph database) and Redshift, designed for data warehouses. For in-memory, which is the focus of this document they offer Amazon Memory and Amazon ElastiCache. An in-memory cache is different to a proper in-memory database since database write operations are not stored, which speeds things up if you don’t care about write operations, but also risks losing data due to a server failure. Amazon MemoryDB is a fully functional database, including persistent storage and a transactional log to support failover and recovery.

Customer Quotes

“Amazon MemoryDB for Redis, which delivers microsecond-read and single-digit millisecond response times and handles millions of transactions per second (TPS), has helped the BUD team to quadruple its R&D (research and development) efficiency.”

Steve Zou, Head of Technical Team of BUD Technologies, Pte. Ltd.

“MemoryDB provides low-latency read throughput without compromising on multi-AZ data durability to deliver the highest quality of service.”

Nabeel Ali Memon, Director of Technology – Intigral

“We are impressed by MemoryDB’s high throughput and low latencies. MemoryDB has enabled us to have a durable primary database with the performance benefits of Redis.”

Patrick Sandquist, Engineering Lead – Twilio Segment

The Amazon MemoryDB is a high-performance database that offers scalability by adding nodes, and supports sharding (splitting a large database into smaller units). It is a non-relational NOSQL database, storing data entirely in memory. It is compatible with Redis (REmote DIctionary Server), an open-source, in-memory key-value noted for its high speed. Redis was once just a cache used in conjunction with other databases but these days has its own data persistence. This is a useful feature since Redis was one of the very first in-memory databases, and it means that, amongst other things, Redis applications are very easy to port to Amazon MemoryDB.

Amazon MemoryDB runs on the Amazon AWS cloud platform. As well as regular database features, it has a distributed transactional log that allows for high availability and rapid recovery if need be. Data is encrypted in transit using the Transport Layer Security (TLS) protocol, ensuring that data is secure. The database allows multi-region replication, with very high availability and low latency for situations where data needs to be distributed across geographic reasons e.g. for regulatory purposes. It is a modern product based on a microservices architecture, allowing for easy integration into applications. Pricing is usage-based. This database is not a columnar database but instead stores data as key values, and supports graph and document storage. This means that it is not optimised for analytic use cases (like columnar) but instead is aimed at ultra-fast real-time applications with low latency. It does provide native JSON document support.

Some applications are very demanding and require ultra-fast performance, for example real-time fraud detection or some on-line gaming applications, financial trading systems or caching layers in microservices architectures. If performance is the priority, then an in-memory database provides the fastest database performance, quicker than that of conventional relational databases that use disk storage. Because Amazon MemoryDB is cloud-based, it scales easily by just adding more compute power. Clearly this all comes at a price, since memory is much costlier than disk, but for some applications, high performance is the priority even at a cost.

The bottom line

Certain applications demand ultra-high performance in real-time, such as some financial trading systems or fraud-detection systems. In such cases an in-memory database will likely provide the ultimate in performance, albeit at a price, since memory is much costlier than disk storage. The nature of Amazon MemoryDB means that it may not be suited to all use cases, such as analytics, where a columnar approach may be better, but for some use cases it will be a strong performer. Its compatibility with Redis also has advantages for those who have existing skills or application using that open-source product. Given the cost of in-memory databases, it is important that you carefully consider your specific use case and decide whether it is appropriate, as well as evaluating alternatives.