The return of the associative database?

Philip Howard

Written By:
Published: 1st July, 2014
Content Copyright © 2014 Bloor. All Rights Reserved.

In May 2013, the US Navy put out a requirements document for the development of an open source, US-based, “scalable, secure associative database”. You probably missed the announcement. Three companies were short listed to fulfil the Navy’s needs: Applied Technical Systems Inc., Boston Fusion Corporation and Systems & Technology Research. The first of these was awarded the contract.

There are a couple of interesting things here. The first is the fact that the Navy wants an associative database in the first place. Why? The proposal document is littered with references to graph databases, triple stores and the like and it’s not as if there aren’t any open source graph database projects, though they might not be as secure as the Navy requires. In any case, why not go for graphs? Well, actually, associative databases have one crucial advantage over graph databases, which is that while the latter is about entities and relationships between those entities, an associative database not only encompasses associations (aka relationships) between entities but also associations between associations. For example, in a graph database expressing a nuclear family you might have father related to mother, father related to son and mother related to son: that’s three entities and three relationships. In an associative database, on the other hand, you have a 'child of' association that associates with the mother-father association. In other words you still have three entities but only two associations.

This may not seem like much of a difference but suppose you have 100 guests at your wedding then each guest has exactly one association with the wedding as opposed to one relationship with each of the parties to be married. Thus there are twice as many relationships as associations. And this is before we consider relationships between guests. In other words, associations are much more efficient as relationships proliferate. This can get important when you want to scale to petabytes (and, presumably, still get good performance), as the Navy does.

Associative databases have been around longer than graph databases, at least commercially, but they have been more widely deployed as underlying technology than as platforms. For example, Kalido (now Magnitude Software) uses associative techniques, while X88’s Pandora is based upon an (in-house developed) associative database. The only database offering per se, at least that I am aware of, is Sentences from Lazy Software (so called because it was designed to make software development easy: “lazy”). This enjoyed some success in the early years of this century only to go dormant for a significant period before being revived over the last couple of years.

So, I like the concept of associative databases. However, it is too early to say whether we will see a major revival of interest, not least because Applied Technical Systems, according to its web site, specialises in developing software for the US Navy. This may mean—I hope it does not—that the eventual product does not get marketed more widely. If that is the case then Lazy Software is going to be left to fly the flag largely on its own and, while I wish it every success, that is not going to be easy in the face of the rising tide of graph databases.

Readers' comments

There have been 3 comments:

  1. pl80 said on :

    "Atomic DB is a commercially available associative database."

  2. Philip Howard replied on :

    "Indeed it is - but I only discovered it after this piece was published."

  3. pl80 said on :

    "On a 2nd thought: in a graph database, why not make wedding an entity and have relations between guests and the wedding. That makes it 102 relations vs 101 associations, and 103 vs 102 in terms of entities... Oh well ... still the advantage is on the associative site."

Post a comment?

We welcome constructive criticism on all of our published content. Your name will be published against this comment after it has been moderated. We reserve the right to contact you by email if needed.

If you don't want to see the security question, please register and login.