Neo4j

Historically, Neo4j has prioritised performance over scale but over the last couple of years it has put significant emphasis on scalability. This started with support for read replicas and, in the latest release (3.4), the implementation of full horizontal multi-cluster scaling. Since that release the product also supports native string indexes, which improve write performance; 3D geospatial search; a bulk data loader; security applied against property values; a new date/time datatype; and faster Cypher run-times. Other major developments in Neo4j include support for high performance graph algorithms (thirty of them), support for a variety of types of knowledge graph, and preparatory work to leverage new hardware capabilities such as Intel Optane and IBM Power 9. However, from the perspective of supporting analytics, the most notable recent developments from Neo4j have been in complementary products, most notably Morpheus for Apache Spark and Neo4j Bloom.

Fig 01 Neo4j Bloom user interface

Morpheus for Apache Spark (currently in beta) will allow pattern-based graph analytic queries to be performed directly within a data lake, using a “tables for labels” scheme to automatically map relational views into an in-memory Spark graph. Data can be stored as a graph in Neo4j and easily move between the Neo4j and Spark environment at high speed. On a related note, as part of the preparations for the forthcoming Apache Spark 3.0 release, the Spark development community has just voted to add Cypher-based property graph querying based on DataFrames to Spark. As a result, Spark 3.0 users will be able to use the Cypher for graph query processing, as well as having access to graph algorithms stemming from the GraphFrames project and GraphX.

Neo4j Bloom provides a visualisation and communication interface for non-technical users so that they can explore, edit and search graphs, and create storyboards. Features include natural language search; business views of your graphs (for example, by department or looking at/for sensitive data); the ability to select, expand, dismiss or find paths through your graphs; browsing; code-free graph change capabilities and support for the use of GPUs for high-performance rendering. An example of the Neo4j Bloom user interface is shown in Figure 1.

Neo4j is the clear market leader in the graph space. It has the most users, it uses a widely adopted language (not just by Neo4j but also many other suppliers of graph databases) that is much easier to use than Gremlin and, in many respects, it has consistently been a lot more innovative than its competitors. This is in part because of the maturity of the product and partly because its success has meant that it has the resources to introduce such developments more quickly. It is, in effect, the Oracle or SQL Server of the graph database world.

More generically, graph databases are a natural home for combining operational and transactional processing with analytics. Their understanding of relationships, combined with the fact that is no need for multi-temperature data and no requirement to store the data twice (once in memory and once on disk) makes graphs an obvious candidate for processing in hybrid environments.

The Bottom Line

All analytics involve relationships in one way or another, so graphs that represent those relationships are an obvious way to explore and analyse them. When you bear in mind that Neo4j has spent most of its history focusing on support for operational and transactional processing, it is clear cut that it is worth serious consideration for hybrid processing environments.

Historically, Neo4j has prioritised performance over scale but over the last couple of years it has put significant emphasis on scalability. This started with support for read replicas and, in the latest release (3.4), the implementation of full horizontal multi-cluster scaling.

Also in the 3.4 release the product supports native string indexes, which will improve write performance; 3D geospatial search; a bulk data loader; security applied against property values; a new date/time datatype; and faster Cypher run-times.

Figure 2 – The Neo4j Bloom interface

Unusually for a property graph, SPARQL is supported. So too is Gremlin (part of the Apache Tinkerpop project). However, the emphasis is on Cypher and the company plans to introduce a “Cypher for Gremlin” capability in addition to the “Cypher for Spark” capability that is already available. In the context of the latter the company currently has a product in alpha testing called Morpheus for Apache Spark. This is intended to allow graph analytics within a data lake (Hadoop and Hive) with in-memory graphs, graph storage within Neo4j and high-speed data transfer between the two environments. In the future, Morpheus will support any source supported by the open source Kettle data integration suite. Also, importantly, the company has introduced a GQL (graph query language) Manifesto as a step towards having a common standard for graph databases. This has been proposed to the ANSI SQL committee for approval and is supported by a range of technology vendors including Talend, SAP, Tableau and others. This proposal is running in parallel to the ANSI SQL property graph extension program.

Other major developments in Neo4j include support for high performance graph algorithms; preparatory work to leverage new hardware capabilities such as Intel Optane and IBM Power 9; integration and the introduction of Neo4j Bloom, which was released in May 2018. This provides a visualisation and communication interface for non-technical users so that they can explore, edit and search graphs, and create storyboards. An illustration of Neo4j Bloom is provided in Figure 2.

Neo4j is the clear market leader in the graph space. It has the most users, it uses a widely adopted language (not just by Neo4j but also many other suppliers of graph databases) that is much easier to use than Gremlin and, in many respects, it has consistently been a lot more innovative than its competitors. This is in part because of the maturity of the product and partly because its success has meant that it has the resources to introduce such developments more quickly. Its competitors have historically argued that the product did not scale well but the multi-clustering now available should knock that argument on its head. Some vendors that specialise in analytics will claim that they can outperform Neo4j and this may be valid, but Neo4j does not have this limited focus: it is, in effect, the Oracle or SQL Server of the graph database world. It is not the equivalent of Teradata. In other words, it is a general-purpose graph database, and it is no coincidence that it is the leading product in that space. In this context, we should comment that we have little faith in the various benchmarks published by other vendors; not least because they are not typically comparing apples with apples.

The Bottom Line

Whenever we talk to a vendor in the graph database space it is Neo4j they compare themselves to. Even if they do something different and address a different market, Neo4j is the benchmark. In pretty much every instance Neo4j should be on your short list.