Other things being equal - namely the capabilities and performance of relevant products - arguably the most important factor in deciding to opt for one product or service rather than another is, or should be, the total cost of ownership (TCO) of those products. However, this is something that analyst organisations rarely consider. Instead, we tend to focus on product capabilities: witness our own Bullseye, the Forrester Wave and the Gartner Magic Quadrant.
The reason for this focus on product capabilities is not hard to spot: it is relatively easy to collect the necessary data, since vendors are only too happy to help you. Measuring TCO, however, is another matter, and the biggest problem is ensuring that you are comparing like with like. The major elements of this are reuse, scale and complexity. A tool that might be very good, and inexpensive, for a single project that lasts two weeks would not necessarily be comparable with a product intended for multiple projects, each of which will last six months or more. In particular, there is an issue with complexity: you can't really ask "how complex is your project?" because the answer is so subjective. What you need is a proxy for complexity. Thus, when we have twice previously investigated TCO for data integration tools, we have used the number of sources and targets for each project, as a proxy metric for complexity.
Going beyond complexity, there is also the question of gathering the data. You can ask vendors for typical costs for a configuration of a specified size, but this is not ideal. It only really works when you know that the suppliers concerned offer comparable products. Moreover, you need to apply non-disclosure principles so what you end up with is an average rather than direct vendor comparisons. A second approach is to interview users who have done detailed product evaluations of their own, or have replaced one product with another. But there are a limited number of these. Ultimately, the best method is to survey users but this, of course, is dependent upon encouraging enough respondents to complete the survey.
I have discussed all of this because we are about to launch our most ambitious TCO research programme ever. This is going to encompass a range of data management disciplines, namely data integration (both conventional and to/from the cloud), data quality, data governance, master data management and data lake management. The URL for the survey is https://www.surveymonkey.co.uk/r/BloorTCOSurvey and I'd like to encourage readers that work in any of the areas mentioned to complete the survey. We will make a donation to charity (see http://bloodroadfilm.com/support where there is a link to the Mines Advisory Group, which campaigns for the removal of unexploded mines) based on the number of survey respondents we get. I would also like to ask readers that work for relevant vendors to encourage their users to complete the survey. Needless to say, the ultimate results (six different papers: one for each of the topic areas) will be published and available for free download from the Bloor Research website.