TIBCO Spotfire 5; Analytics starts to grow up in the Enterprise space. - Spotfire Release 5 focussing on Enterprise ready features

Advanced analytics is becoming a mainstream business activity and the tools to enable that to happen are being introduced and improved all of the time. This week I had the fortune to be briefed on a product that I have been aware of for some time, but which I had not formally got to learn about, TIBCO Spotfire. Spotfire Version 5 is due shortly, and one of the reasons I was keen to talk to them is that, unlike so many of the exciting products in this area, this is one that is actually available in EMEA late November 2012, and not at some point to be determined in the future.

Analytics used to be the preserve of a small group of very skilled statisticians, who got paid an awful lot of money and, in my experience, seemed to spend an extraordinary amount of time building anything that gave you any really useful insight. But I would contend that all of that is now changing very rapidly; and with the amount of data being made available to help explain the market, and the operations of all enterprises, this is an area of great growth. In a global market you can no longer rely on intuition to get you through, you need facts. In an era of big data, reports and even slicing and dicing data with OLAP, there is just too much data on hand to make assimilation of data, and its translation into actionable insight, meaningful. We all know that banks, retailers and telco’s have always had a lot of data but, with the evolution of RFID and sensors on all manner of devices, it is no longer just confined to those sectors that the explosion of data is occurring. It is all very well to have data, but how do you make sense of it, and the answer has to be statistical analysis. We have to use tools to help us classify and detect patterns in the mass of data, and identify the significant from the random. As I intimated, it is no longer acceptable to spend months to tell someone something they already half know, they now want you to confirm that half understood fact in a matter of minutes, and then go on to provide fresh insights that stimulates new trains of thought.

So, what is so important about TIBCO Spotfire 5? It is one of the first tools to really tackle how to make discovery of data within the Big Data of an enterprise an enterprise-ready undertaking. What do I mean by that? Well, in the past it has been the case that most of the tools can be used to analyse very large data sets, but they do so in ways that are ad hoc and hardy likely to gain acceptance by your friendly Run and Operate team as something they will willingly schedule into their month end run times as something that is enterprise ready, and ready to be part and parcel of the same world as your average ERP month end activity.

What Spotfire is aiming to do is to provide the capability of the small scale artisan activity, and spot the all-important outlier of data that provides insight, but they want to do it in a way that scales without requiring massive investment, and which run in the sort of time scales that fit in with mainstream activity, and not those of a data scientist. To this end they have rewritten the memory data engine, which has resulted in massive in-memory performance improvements. They have moved from the use of C# to C++, giving them more low-level control over memory, and they have changed the way that run times operate so that instead of running each component discretely, the elements are now built and rendered in one pass, which results in impressive improvement in the time to deliver a visualisation. They use hybrid operation using the power of the database, where appropriate, to perform some functions that are best done within MS SQL, Oracle or Teradata, and then Spotfire ingests the results and makes them readily available in visualisations etc.

I have written before about how Universities are now adopting R as the new standard for all statistical analysis and how if you want to be able to capture that mass of talent R has to be the way to go: Spotfire is now part of that trend with the TIBCO Enterprise Run Time for R (TERR). TIBCO are making things more ready to run with Self Service Predictive Analytics so the everyday tasks of statistical analysis are provided to the analysts to speed up operations.

They are also looking to take this capability and embed it into the more familiar elements of the TIBCO market offering. Embedding analytics into their enterprise tools, affording insight and prediction into enterprise operations to enhance management of resource in all manner of aspects, is surely the way that all tools must go and TIBCO are to be commended for leading in this field.

TIBCO are at the heart of the enterprise architecture and it is good to see that they view Spotfire not just as an interesting outlier to their mainstream business but as an integral part of where things are going. We live in a world in which data and its analysis is key to maintaining competitive advantage, managing cost and understanding operations. Spotfire looks to be at the very core of making that a reality and I can commend anyone interested to take a closer look.