CopperEye launches Greenwich

Regular readers will know that CopperEye started life as a specialist vendor of indexing technology. That simplifies things a bit but basically it provides faster and more flexible indexing to run against Oracle and Informix databases (it co-developed a Datablade for the latter in conjunction with IBM). However, the indexing techniques it has developed are equally applicable to flat file systems and in practice CopperEye has had most of its success with the application of its indexing techniques to flat file environments, such as call data records for telecommunications.

Greenwich is a new product that extends this existing success to any environment where the data is written once and never changed. Apart from call data records this would include RFID data, credit card transactions, data that needs to be stored for compliance and a variety of other types of information, which are generally categorised as event data.

Now, in this sort of environment you don’t need a relational database to store information. Why have all that complexity when the data is never going to be updated? So a flat file system makes sense. Indeed, I wrote recently about Sleepycat and its Berkeley DB product, which addresses a similar market and which is based on an ISAM (index sequential access method) architecture. However, what Berkeley DB doesn’t have is the sort of performance that you should be able to get from Greenwich. As an example, CopperEye recently reported that at the recent IDUG (International DB2 Users Group) conference it was able to demonstrate query support on a 1.7-billion-row set of RFID data while handling 13,000 transactions per second on a two-CPU Linux server.

Another difference is that with Sleepycat you explicitly need Berkeley DB to store the data in (which also means it has to be moved there, which in turn means that you have to buy a suitable tool for the purpose). With Greenwich you just leave the data where it is.

The way that Greenwich works is non-intrusive and it does not alter, move or delete your data files – it just discovers them and then builds its own index files which you can then use to query the data. In the latter case, the Greenwich software presents the data through relational views in a conventional manner so that you can access that data from a relational database environment via ODBC or directly from a business intelligence tool. In either case, access is via standard SQL.

In a nutshell, the use of Greenwich means that you do not need a relational database (or any other sort of database) in which to store static event data and this will save you a lot of money. At the same time you should get significantly better performance from Greenwich than you would do if the data was in a relational database – which sounds like a win-win to me.