The game is afoot in the data quality jungle

A lot is happening in the data quality world: Identity has announced a partnership with Siperian; we will shortly be seeing announcements from major vendors in this space; and Trillium (not to say that Trillium is not a major vendor) has announced TS Insight, a data quality dashboard that complements its existing products.

Now, for those of you that are familiar with Trillium, you will know that TS Discovery already has a dashboard. The difference, however, is that TS Insight is designed for use by data stewards, business managers and executives rather than IT managers, as the details it provides are more business-focused.

I expect that existing users of Trillium will find TS Insight very useful, especially when it is used to help data governance processes. However, one drawback of the system is that you can’t drill-down from it. Instead, you have to open Discovery, go to its dashboard and drill-down from there. I’m not sure that if I was looking for a new data quality product for the first time that this is what I would want. Indeed, it contrasts poorly with, for example, Datanomic’s dn:Dashboard.

Datanomic was also recently in the news: it released version 6.0 of dn:Director at the end of 2006 and the point about this is that it is a single product with a single repository that spans all areas of data quality provision: not just analysis and profiling, cleansing and matching but also parsing and phrase profiling. Now dn:Dashboard sits on top of dn:Director in much the same way that TS Insight sits on top of TS Discovery and it does much the same things. But the difference is that you don’t have to open up a second product if you should want to look at details in depth.

Actually, I’m quite impressed with dn:Director. Datanomic has always had the idea that you shouldn’t separate profiling, analysis, cleansing and so forth and that they are all part of a whole; and that you shouldn’t be dictated to by the software making you do things in any particular order or combination. However, that was always a difficult sell when the company was offering separate products for each of these functions. But with this release all of those separate products have disappeared and there is now only dn:Director and the optional dn:Dashboard.

Actually, there’s something else that’s cool about dn:Director. Around a year ago the company bought another vendor called Tranato, which specialised in parsing and phrase profiling, whose facilities have now been built into dn:Director. Briefly, parsing allows you to analyse free-form text so that you can support product cleansing for catalogue suppliers, which is something that is an area of weakness in most (non-specialist) data quality products. Phrase profiling, on the other hand, allows you to search for words or word groups. In particular, it supports the concept of street furniture: lampposts, for example, don’t have addresses, but highways authorities need to know where they are; as does anyone else that looks after and maintains such equipment, such as telcos, utility companies and the like. Now, it may be that every data quality vendor supports street furniture but in all the years I have been looking at such products I do not recall ever having a supplier mention it to me, which suggests that they don’t.

There is lots of activity in the data quality market right now, and no doubt the industry big boys, IBM, Trillium, Informatica, Dataflux (SAS) and Business Objects will continue to dominate the market but there are a number of smaller vendors, such as Datanomic, that are worth serious consideration.