Improving on optimisers

After all these years you would assume, wouldn’t you, that database optimisers were pretty good? Companies like IBM and Oracle have been costing query plans and re-writing SQL not just for years but for decades, so you would expect that they would know what they were doing.

Well, if that’s the case then how come that ActiveBase and its UK distributor Application Performance, can making a living out of selling the former’s ActiveKnowledge product, which has been expressly designed to improve optimiser performance though, to be fair to IBM, only for Oracle?

ActiveKnowledge acts like what you might describe as a SQL Firewall. That is, it intercepts all incoming SQL, does various clever things with it, and then passes it on to the database to process. What, of course, is important is the “clever things”. So, what are these?

The main “clever thing” that ActiveKnowledge does is to assume that the optimiser doesn’t know how to calculate costs properly—indeed, that an optimiser can’t, by definition, calculate costs accurately. The company’s view is that costs can only ever be estimated and that those estimates will always include inaccuracies which, in turn, will result in queries running more slowly than would otherwise be the case.

ActiveBase contends that the only way that you can accurately calculate the costs of a query are to run the query multiple times using multiple options and actually find out in practice which is the best way to run a particular query. This is exactly what the product does: determines multiple execution strategies and then benchmarks them. For obvious reasons, therefore, the product is most suitable for environments where you have many queries that are repeated on a regular basis.

Following on from the determination of the most efficient way to run any particular query you can then derive a rule to correct the execution path, for this query (and any other queries that are similar to it) which is then activated and the improved query is passed to the database whenever this query arises.

Other facilities provided include SQL rewrite, support for hints, blocking rules (for example, blocking Cartesian joins), query delays and so forth. In addition, Application Performance is developing rules packs for specific application environments such as Chordiant, Siebel and so on. There is also an auditing module that allows you to inspect usage information in a graphical fashion, with details such as the number of reports submitted daily, average query times, and so forth.

Of course, the real proof of this particular pudding will be to see it improving the performance of your queries, which Application Performance will be happy to demonstrate in a proof of concept. Existing customers have reported a variety of results, ranging form an average performance improvement of 30% to some queries with as much as an 80% performance improvement. At one Siebel installation the company was able to bring down elapsed time for one query from over 10 minutes to 30 seconds.

These results highlight another consideration: this is not just for queries that you run regularly, it is also for queries that run in a relatively short period. This should be obvious: running benchmarks for queries that take days would typically be too onerous (except on a test system). Nevertheless, improving the performance of all those short queries releases resources for those longer running enquiries so this should produce a spin-off benefit.

In a nutshell: it is certainly worth considering if you are an Oracle user.