Bringing clarity to the data warehouse and appliance market

The data warehousing market is awash with claim, counterclaim and confusion. In particular, what exactly is an appliance? What does it do? How is it distinguished from a virtual appliance? Is a virtual appliance actually a sensible concept? Regardless of the answer to those questions, where does an appliance fit in? When should you be considering an appliance and when should you be considering a more conventional approach to data warehousing? Are appliances limited to fast table scanning and data marts or do they have wider applicability? And it’s not just about appliances: where, for example, does SAS Intelligent Storage fit in, or Sybase IQ?

All these are questions that you might reasonably ask. And you won’t get any very sensible answers: at least, not from vendors, because they all have their own particular axes to grind. For example, Teradata wants to pigeonhole appliances as non-threatening data marts but wishful thinking doesn’t necessarily make it so. Similarly, the true appliance vendors attack the virtual appliance vendors on the basis that, for example, you can’t have a virtual fridge. Fair enough. But it’s also possible to decide that if it quacks like a duck and waddles like a duck then it probably is a duck.

I could go on: there are performance claims and counterclaims, happy versus unhappy user stories, “we beat you in this proof of concept – no, you didn’t – yes, we did” arguments, and so on and so forth. There are already put-downs and rumours circulating about some of the vendors that have yet to come to market, and this is only likely to get worse as more suppliers introduce their products.

Incidentally, while on the subject of new entrants, both Vertica and Dataupia have recently received venture capital funding. While Dataupia has still not announced what it is developing, the Boston Business Journal, which reported the funding, suggested that the company might be intending to enter this market. Unfortunately, however, it quoted me as the source of this information and as I was only making an intelligent guess when I made that suggestion it doesn’t take us much further forward. That said, one of those rumours I mentioned before suggested that it isn’t an appliance at all. Anyway, I have a meeting with Dataupia later this week after which I will be able to reveal all (or not, if the meeting is under non-disclosure).

Anyway, back to the subject in hand: where do the appliances fit as compared to traditional data warehousing? Good question. Not an easy answer. Of course, it depends on what you want to do. If you need to have active data warehousing (and don’t have another way to do it—consider the possibilities of complex event processing—an article for another time), if you have lots of complex joins to do, if you want to use the warehouse as a hub for master data management, or you want the convenience of a pre-built business model, then these might all be good reasons to elect for a traditional approach. But how much should this be offset by the cost and performance advantages that appliances can offer elsewhere?

There is clearly a lack of clarity in this space and we need to cut out the marketing guff and focus on what actually matters for users: does the product do what the user requires of it, for a price that he’s prepared to pay? The question then becomes one that is about the capabilities of appliance products (which may, of course, differ) as opposed to enterprise data warehouses. Now, an individual user may be able to work that out for himself because he has a specific set of requirements but that doesn’t constitute anything more generic. Equally, I may think that putting down my ideas on the subject can answer all of these questions, but that is after all just one person’s opinion and I think that the market actually needs something more definitive.

For this reason, Bloor Research is launching its new series of conferences with a conference on data warehousing and the role of appliances, which will be held on November 2nd in central London. We are going to take a rather different approach for this. I will be writing a paper prior to the conference (upon which I will consult with vendors of all hues and stripes; and anyone else who cares to contribute—I would be glad to hear from readers), which will highlight the issues involved and this will be circulated widely to both attendees (entrance is free) and others prior to the event. After both user and vendor presentations, the conference itself will conclude with a panel discussion (and audience questions) at which, hopefully, some consensus will be reached as to the role of appliances and whether it is worth distinguishing between real and virtual appliances, and what the position of IBM’s BCU is, and so on. Subsequently, I will write a further paper that will summarise these conclusions, which will again be circulated to attendees, while a précis will also appear on this website.

While the conference should hopefully be entertaining, interesting and educational the main purpose of the exercise is to bring clarity to the market and the wider the canvas of opinion the greater that is likely to be. At least then we can stop having all these silly arguments.