Figure 1 – Striim’s user interface
Striim provides a web-based graphical, drag-and-drop development environment. Through this user interface, illustrated in Figure 1, you can capture data from many sources such as databases via CDC (Change Data Capture), log files, message queues (Kafka, RabbitMQ and others), events, and so forth. You can then build transformation and analytics pipelines, and connect the results to dashboards, external storage, and messaging platforms. A preview capability is provided for test and debugging prior to the deployment of the application on the operational platform.
Pipelines, an example of which is shown in Figure 1, are built using Striim’s proprietary SQL-like language (TQL). TQL runs in-memory and, in addition to filtering, transformation and aggregation of data, can be used to enrich streaming data with reference data stored in the built-in distributed in-memory data grid. There is also a facility for users to add extensions written in Java, and the platform supports the import and export of Java analytic models (although not models using R or Python). Nor does the company support PMML (predictive modelling mark-up language). Tumbling, sliding, and session windows are supported based on record count, time or data attributes, with support for filtering and aggregation, joining streams and historical data, pattern detection, and there is a library of predictive analytics functions. A wide range of connectors are provided, although beyond MQTT and OPC UA (both IoT protocols) you may need to work with one of Striim’s partners for IoT sensor connectivity.
As far as dashboards and visualisations are concerned, there are a number of noteworthy features. For example, Figure 2 shows the ability to filter (either by time or field) real-time data. The ability to rewind time-based queries to look at past data is also provided, as is page- and chart-level searching and filtering. Striim charts may also be embedded into other HTML pages.
Figure 2 – Striim dashboard and charts featuring filter capability
The Striim platform is highly scalable and has an elastic architecture. The distributed execution platform combines a continuous query engine, in-memory data grid, high speed messaging system, and a results cache built on Elasticsearch. Incoming data streams are sharded over the cluster for horizontal scalability, with checkpointing provided for recovery and restart from the last known good state, providing exactly once processing (E1P) guarantees. There are also significant integrations with cloud providers (including features such as multi-threading), notably with Amazon (AWS and Kinesis) and Microsoft Azure, and there are also specific facilities to monitor Kafka environments.
