EsperTech

Figure 2 – Esper architecture

Esper is a platform for the analysis of real-time data, which may or may not be combined with historic data. Processing is developed using EPL (event processing language), which is based on SQL but has extensions to support pattern (and rules-based) matching. There is, however, a presumption that this is based on patterns developed using EPL as opposed to those that may have been developed in a third-party tool and then imported, though there is an API for Java integration.

Figure 3 – Dashboard built using EsperHQ

Apart from EPL there are a number of other facets of Esper worth describing. Most notable is the support for Apache Kafka. This isn’t just for input and output adapters, as illustrated in Figure 2, but Kafka is also used as the underpinning for horizontal scalability in the Enterprise Edition, with Kafka Streams Processors running in each distributed instance and managed by Apache Zookeeper. You can add new instances on the fly and this architecture means that if one instance goes down the others will take over from it. EsperHA, on the other hand is mainly about preserving state in the event of a failure. This is important when you have linked events and you need to remember that the first event has arrived (but not the second). Finally, it is also worth mentioning EsperHQ, illustrated in Figure 3, which provides a graphical user interface for run-time monitoring and event-stream visualisation.

The ability to do real-time streaming analytics is becoming increasingly mission critical for many organisations. While this has always been true in capital markets, where EsperTech initially focused, it is now also true in many other areas, including energy, logistics, manufacturing and others. EsperTech has proven credentials for providing high throughput event processing combined with low latency, and is well placed to continue to serve this community going forward.

The Bottom Line

EsperTech sees its product(s) as parts of a larger stack as opposed to a one-stop shop solution. Thus, Esper and NEsper will be of most interest to third-party vendors that want to embed a complex event processing solution into their platform, as well as to end user organisations that have existing Kafka installations and require complex event processing, or require a ready-to-run CEP server, or that are intent upon building their own streaming analytics capabilities on a best-of-breed basis.

Fig 2a - Esper architecture

Esper offers push-based, loosely-coupled analysis of real-time and historic data, providing sliding analysis rather than static querying. Processing is developed using EPL (Event Processing Language), which is based on SQL but compiles to bytecode for Java or .NET (meaning it is not an interpretive language). It also has extensions to support pattern (and rules-based) matching. There is, however, a presumption that this is based on patterns developed using EPL as opposed to those that may have been developed in a third-party tool and then imported, though there is an API for Java integration.

Fig 2b - Esper architecture

Apart from EPL there are a number of other facets of Esper worth describing. Most notable is the support for Apache Kafka. This isn’t just for input and output adapters: Kafka is also used as the underpinning for horizontal scalability in the Enterprise Edition, with Kafka Streams Processors running in each distributed instance. You can add new instances on the fly, and this architecture means that if one instance goes down the others will take over from it. Both types of Kafka usage are illustrated in Figure 2.

Fig 3 - Dashboard built using EsperHQ

EsperHA, on the other hand is mainly about preserving state in the event of a failure. This is important when you have linked events and you need to remember that the first event has arrived (but not the second). Finally, it is also worth mentioning Esper Enterprise Edition, illustrated in Figure 3, which provides a graphical user interface for run-time monitoring and event-stream visualisation.

The ability to do real-time streaming analytics is becoming increasingly mission critical for many organisations. While this has always been true in capital markets, where EsperTech initially focused, it is now also true in many other areas, including energy, logistics, manufacturing and others. EsperTech has proven credentials for providing high throughput event processing combined with low latency, and is well placed to continue to serve this community going forward.

Its open-source nature is a particular differentiator, which, though hardly unique within the streaming space as a whole, is significantly rare within the subspace of streaming analytics (as opposed to stream processing, which encompasses Kafka, Flink and so on). It also prides itself on its extensive documentation and self-help.

The Bottom Line

EsperTech sees its product(s) as parts of a larger stack as opposed to a one-stop shop solution. Thus, Esper and NEsper will be of most interest to third-party vendors that want to embed a complex event processing/streaming analytics solution into their platform, as well as to end user organisations that have existing Kafka installations and require complex event processing, or require a ready-to-run CEP server, or that are intent upon building their own streaming analytics capabilities on a best-of-breed basis.