Cray

Figure 1 – The Cray Graph ecosystem

The architecture of Urika-GX is illustrated in Figure 1. As can be seen there is a combination of Hadoop, Spark and graph technologies in a single configuration. There are various points to make. Firstly, an advantage of including Hadoop and Spark is that you can leverage either to transform and load data into the graph database much more efficiently than would otherwise be the case. More generally, Hadoop/Spark are typically used for batch processing purposes. Additionally, Spark can be used for streaming and iterative analytics as well as machine learning, with the graph engine used for graph-related analytics. You can have workflows that span these environments so that different elements of a workflow process can use the technology that is most appropriate for it. Needless to say, you can have multiple processes running concurrently in each mode.

The Urika-XC software package for Cray XC50 contains many of the software packages found on the Urika-GX, with the exception of Hadoop and HDFS. However, it adds additional packages for Deep Learningincluding Apache Spark, Intel BigDL, TensorFlow, Cray ML Scalability Plugin, Java, R, Scala, Anaconda Python and Python, Dask and Dask Distributed, and Jupyter Notebooks. More such support is planned.

Cray sees the necessity for supercomputing to underpin graph and artificial intelligence in many environments, and the Cray Urika suites (Urika-GX and Urika-XC) are just one part of its strategy in this space. The product is specifically targeted at the most intractable analytic problems and use cases include cybersecurity, the Internet of Things, machine learning, research into new drugs and new uses of existing drugs, and to uncover risk and compliance issues within financial services environments.

As you might expect, performance is a key differentiator for Cray and it recently completed a trillion triples benchmark. It is not the first company to do so – there have been at least two others – and it is difficult to compare different systems running different hardware. Nevertheless, Cray’s performance for both inferencing and query response was approaching an order of magnitude better than its nearest competitor.

The Bottom Line

There is a perception that Cray – and, indeed, any supercomputing product – is expensive. However, if you have to process hundreds of billions of triples, and do that within a reasonable timeframe, then that is not going to come cheap. In practice, Cray believes that it is competitively priced compared to other offerings – if there actually are any – with similar capabilities. We are inclined to agree with the company.