TIBCO

StreamBase is a high-performance platform for rapidly building and deploying real time streaming analytics applications. It enables professional developers and business users alike to build these applications using a browser-based, low-code, visual interface. Creating an app is as simple as dragging and dropping components onto a canvas and arranging them into a model. The chief advantage of this system is the ease of use it affords, resulting in a reduced time to develop and deploy applications, and ultimately in a faster time to market. Additionally, citizen development is a real possibility for StreamBase, with the potential to enable self-service streaming app creation and reducing time to market even further. StreamBase also provides a variety of deployment options, including several cloud environments such as AWS and Azure, and deployment as a Docker image.

A wide variety of Accelerators for StreamBase are freely available via the TIBCO community. Each one consists of a collection of pre-built components that have been optimised for a particular use case. Examples include financial fraud, case management, and insurance pricing. Notably, a number of accelerators are built to support the Internet of Things (IoT), including the Connected Vehicles Accelerator, and the IoT Drilling Accelerator. Combined with integration with Jaspersoft for embedded BI and analytics, and the option to deploy on the edge, this allows StreamBase to perform well in IoT environments.

Figure 2 – A LiveView Web dashboard

Although StreamBase integrates with several other TIBCO products, its integration with Spotfire, TIBCO’s general analytics platform, is particularly strong. This integration is focused primarily around predictive analytics and machine learning, due to their synergy with streaming analytics. It is enabled by the Artifact Management Server (AMS), a shared environment for storing and managing assets across both StreamBase and Spotfire. These assets primarily consist of predictive models, created in Spotfire or otherwise (for example, via Statistica), that can then be deployed into StreamBase. StreamBase supports models written in Java, Python and R (via TERR), as well as PMML (Predictive Model Markup Language) and SparkML.

TIBCO Live Datamart is another special case. Technically, it is a separate product. However, it is built on top of StreamBase, and adds significantly to its capabilities. Specifically, Live Datamart is an in-memory data warehouse equipped with a continuous query processing engine. This engine drives alerts and user actions, as well as decision rules that can automatically take action under specified conditions. These rules can be authored in either StreamBase itself or in Microsoft Excel and are stored in AMS. Moreover, Live Datamart provides real time, live visualisation of your streaming data that can be accessed and assembled into dashboards via LiveView Web, a thin web client and dashboard builder for Live Datamart.

Streaming analytics products, such as StreamBase, allow your data to be analysed in real time, as or before it is processed and stored. This gives you the information you need to make informed decisions quickly, resulting in timely decisions based on fresh, current data. This addresses the problem of ‘decision latency’ (wherein analysis is delivered after the optimal timeframe for acting on it has passed) and can result in faster decision making, improved operational efficiency and a better, more responsive customer experience.

StreamBase itself is remarkable for adopting low-code design principles in order to make it easy and fast to build streaming applications, improving accessibility and enabling self-service and citizen development. In doing so, it significantly reduces time to market (and therefore time to value). The selection of available Accelerators adds to this capability. Moreover, integration with other TIBCO products makes StreamBase particularly well suited for use as part of a comprehensive analytics platform.

The Bottom Line

By itself, StreamBase is an effective way to make it easy to build streaming applications. With Live Datamart, it is a complete (and very competent) streaming analytics platform. Furthermore, integration with TIBCO’s wider connected intelligence ecosystem allows StreamBase to form part of an end-to-end, enterprise level analytics solution.

Fig 01 - A model flow in TIBCO Statistica

The core of model management in TIBCO Data Science is the model flow. As seen in Figure 1, this is essentially a form of workflow, created within Statistica, a component of TIBCO Data Science, using a no-code GUI. These flows contain data inputs, models and outputs (shown in Figure 1 in green, orange and purple, respectively) as well the standard flow controls. Business logic can be implemented within a workflow via a rules node, and the flows themselves can be saved as templates and reused inside other model flows. The latter capability is particularly valuable because it allows dedicated data scientists to create complicated, in-depth models and flows, before allowing business users to situate them within the appropriate business processes.

Models and model flows are centrally stored and managed within the platform and accessed via a file-tree structure. Models are versioned, with all previous versions stored within the platform, available to be accessed, deployed or reverted to, as necessary. Third party models (that is, models not created within the data science platform – all standard languages are supported) can also be imported into the platform.

Models and flows held within TIBCO Data Science can be deployed across the broader range of TIBCO products. For example, you could publish your models to the Artifact Management Server, a common model repository, to expose them to TIBCO Streaming (formerly known as StreamBase) for use with streaming analytics, or send them to Spotfire to drive data visualisations. You can also monitor and retrain models en masse via the Monitoring and Alerting Server (a component of TIBCO Data Science Operations), and models can be exposed for external use via OData.

TIBCO Data Science also features a number of collaborative capabilities between data scientists and business users that come into play with model management. Models and model flows (as well as other objects, such as projects) can be shared and worked on collaboratively across the platform. Moreover, models and model flows can be parameterised and tailored to business users before being exposed to them. This might include only exposing a subset of the model’s parameters, renaming fields to make them more easily understood, setting default values, and so on. The aim is to expose all the functionality that a business user needs, and no more, thereby making their interactions with the model as simple as possible. Once a model has been tailored in this way, it can be saved as a template and shared as needed. This is also tightly integrated with TIBCO Spotfire so citizen data scientists can run data science workflows directly from a Spotfire dashboard.

TIBCO Data Science is a platform for analytics in the broadest possible sense. It’s undeniable that machine learning, AI and data science are becoming ever more popular and important parts of analytics, and as a result, if you are leveraging analytics you are almost certainly leveraging analytical models. The sheer amount of these models is growing: we expect large organisations to be utilising thousands or even tens of thousands of models in the near future, if they aren’t already. At that quantity, model operations and management is essential, and TIBCO provides it along with all of its other capabilities.

What’s more, as modelling becomes more common and more popular, data scientists will be more needed (and therefore more in demand) than ever. While TIBCO (or other product) can eliminate the need for data scientists, its collaborative capabilities do enable more effective and efficient use of the data scientists and citizen data scientists you have, by enabling business users to take a larger role in the modelling process. This maximises the value of your data scientists, while providing a measure of self-service for your business users.

The Bottom Line

Model management will soon be essential for any serious analytics solution. TIBCO Data Science not only has it, but integrates it with a wealth of other analytics tools and products.

From a time-series perspective Spotfire works on the basis of time windows (various options) that you define. Specifically, it provides high level operators called Aggregate, Pattern, Join, Query and Gather, which can be combined and sequenced as required. Some 58 aggregate functions are built into the product, including both basic analytical functions (mean, median, standard deviation and so on) and more advanced statistical functions (slope, intercept, correlation, exponential moving average and so forth). A Java aggregate function API is available for users to define their own aggregate functions, if desired.

The Pattern operator has its own sub-language for sequential, value-based and temporal patterns, to which Boolean logic may be applied. Patterns within streams may also be detected using one or more operators in combination. The Join and Gather operators join streams either as two-way or multi-way joins respectively. Finally, the Query operator allows you to join a stream against a relational table, so that you can combine real-time and historic data. Streams may be buffered in such a way that arrival order is not material to the results of the operation so long as emission order is not material either.

Fig 02 Mapping the human body

Of course, for many IoT and industrial applications support for time-series analytics is not enough: you also need to combine this with location intelligence and Spotfire provides significant capabilities in this regard, supporting both geographical and non-geographical (see Figure 2) maps. Multi-layer mapping lets you zoom into successive levels of detail, for example, from the country to the state, county, city, neighbourhood, and house level. Moreover, you can choose to see whatever associations are relevant to you, with data specific to the level you are working at. The product provides worldwide address-level geocoding as well as route calculations with step-by-step directions.

More generally, the product uses native connectors, not just for its analytics but also for data preparation (TIBCO calls it wrangling) so that you can connect to and blend data from  a variety of relational and NoSQL databases; and to cloud environments like Amazon Redshift, Databricks, RDS, Microsoft Azure SQL Database, Google Analytics, and Salesforce.com. You can also build your own custom connectors. Natural Language Query capability – which is used throughout the product – lets you search for any data or connector.

From a more general standpoint, Spotfire supports AI/ML and predictive models developed in other TIBCO tools such as Statistica, as well as supporting languages such as R, Python and Java, and models developed in third party environments such as Spark MLlib or H20 models, or which can be imported via PMML (predictive modelling mark-up language). These can be scored on streaming data by operators in Spotfire. Also notable is the TIBCO Artefact Management Service, which supports governed deployment for these models. Once approved, then can be pushed out to Spotfire applications, or the applications can be set up to query for updated models at regular intervals, or under certain conditions. These models can be updated at runtime without requiring any application downtime.

Spatial analytics is commonplace and all the providers of streaming analytics platforms have the ability to process streaming data via time windows, in some sort of similar fashion to TIBCO Spotfire. In addition, there are lots of products that enable you to visualise time-series data. What is rare – even to the point of almost non-existence – is to find an analytics platform that has any sort of sophisticated functions for analysing historic (stored in a database) time-series data, whether as a stand-alone function or in conjunction with real-time data. There are specialised tools that focus only on time-series data but we know of no other general-purpose tool, apart from TIBCO Spotfire, that has this ability.

The Bottom Line

TIBCO Spotfire appears to be unique. Not only have we been unable to find any product with comparable capabilities but none of the database vendors with time-series capabilities could point us in any different direction. We don’t think anything more needs to be said.