Goodbye CEP, hello streaming and Cassandra

Written By:
Content Copyright © 2013 Bloor. All Rights Reserved.
Also posted on: Accessibility

You may have missed it but Progress has sold its Apama division to Software AG and StreamBase has been acquired by TIBCO. This means that there are now no significant independent vendors of complex event processing (CEP) products. Now, Software AG is about integration and middleware while TIBCO, notwithstanding its ownership of Spotfire, is pretty much about the same things. No doubt both companies will continue to play in the capital markets space but I expect both of them to drop any pretensions to being general-purpose analytic platforms for high velocity data. Certainly, I don’t see either of them competing very successfully with either SAS or IBM for general-purpose environments, which is probably why the sellers were happy to sell.

Since neither SAS nor IBM refers to its respective product as complex event processing we may as well kiss that term goodbye: (event) streaming is where it is now at. However, it is worth considering whether event streaming is always the best approach. In particular, advocates of Cassandra would suggest that event streaming is fine if you just want to analyse what’s happening right now but that if you want to understand what’s happening now in the context of what was happening five minutes ago and five minutes before that – in other words, trending – then Cassandra is better option.

And, of course, there’s Impala, which is being touted as real-time Hadoop. Well, only if you conveniently ignore “near” or even possibly “far”. Cloudera Impala allows “real-time queries against data stored in HDFS or HBase” and, it is claimed, is an order of magnitude faster than Hive. Good. How long does it take to store the data in Hadoop? If we are talking about scads of high velocity data this isn’t going to compete with either streaming or Cassandra. It’s good, I like it, but don’t confuse it with either of these two, which have been engineered from the outset for real-time analytics against streaming data.

Of course, the other problem with streaming platforms, as opposed to Cassandra, is that they are very expensive. This has limited the market opportunities of companies in this space, more or less, to fraud, security and capital markets plus a few isolated use cases. Cassandra, it seems to me, has much greater potential to grab market share and until and unless the big boys with the big streaming platforms bring their prices down dramatically, what we are going to see is more and more Cassandra implementations, which must be good news for DataStax and Acunu.

This Post Has 3 Comments
  1. I should have mentioned SQLStream alongside IBM and SAS as a stream processing platform for operational analytics.

    1. Obviously a SAS salesman! SAS has actually been behind the curve on streaming. I asked the company’s previous CTO, Keith Collins, about the company’s plans for CEP a number of years ago, and he pretty much dismissed the technology. If memory serves me correctly it wasn’t until well after IBM came out with what was then called System S (now InfoSphere Streams) that SAS genuinely entered this market. To be fair, I know that SAS had been doing some work with fraud detection (for example) long before it formally released a product but, nevertheless, the company was relatively late in coming to market. This does not, of course, mean that the company can’t lead in real-time analytics, but it does make it more difficult. Moreover, there’s lots of competition: Spark Streaming, Apache Storm, SQLStream, IBM, Oracle, SAP, Microsoft, all the ex-CEP vendors and lots of niche players. Everybody is in this market because it is core to the Internet of Things. I don’t think you can say that SAS is anything like leading this market. In fact, I don’t think we are going to be able to clearly identify a leader in this market for a number of years, as it is evolving rapidly.

Comments are closed.