Provenance the next step in Governance
Governance is the term used to describe all the processes that should be in place in an organisation to ensure that it is run as the management require. Some of the governance rules will be to ensure the organisations compliance with external regulations.
At present much of the effort in this area is concentrating on how to define the governance required and how to automate it. However, in the coming years the emphasis will move towards being able to demonstrate to management, and critically to external auditors, that the rules have been followed and the resulting data, information and artefacts are correct. The ability to prove the verisimilitude of the data can be compared with provenance in fine art; the provenance of a work of art is the trail of information that shows that an artefact is the real thing.
In late 2004 the European Union set up a research project to investigate Provenance, with funding from the European Community’s Sixth Framework Programme. The project will complete in August 2006 with a final release of code to implement provenance and a proposed standard. As a step in that process the project recently published an ‘Architecture for Provenance Systems’ which I think should be read by anyone developing architectures or solutions for governance projects.
The basic concept is that, if provenance is going to be provable, the organisation must document the processes as they take place, so that every step in the development of some data or artefact can be verified. Not only must running processes be documented, they must be documented in a way that will be accepted by external auditors. In reality this means the recording must be transparent and external to the process and the recording must be held in an external and trusted provenance store. Once it is in the provenance store there need to be tools that can report on and analyse the information in the store and hence provide information on the provenance trail.
The architecture defines the basic interfaces between the source systems and the provenance store, the method of storage of provenance information and the interfaces for analysing and reporting information in the provenance stores. To support these functions the architecture has to discuss, amongst other issues, security and distribution. Security is obviously vital as any loss of integrity of the provenance trail, or its link with the base documents, would completely invalidate the whole concept. Distribution is also essential as most provenance trails will cross multiple internal and external organisational boundaries and there has to be a way for the auditing functions to see across multiple separate provenance stores.
The project is validating the concepts by looking at two very different business scenarios:
- The provenance of simulations in the aerospace industry where the results are very big and may need to be kept for up to 99 years.
- The provenance of information on organ transplants where many different organisations are involved in the process and the issues of patient confidentiality and safety are key.
The experience gained should prove that the architecture is fit for any other provenance requirement.
The project is being run by a consortium of vendors, academia and users and consists of:
- IBM United Kingdom Limited, United Kingdom
- University of Southampton, United Kingdom
- University of Wales, Cardiff, United Kingdom
- Deutsches Zentrum fur Luft- und Raumfahrt s.V., Germany
- Universitat Politecnica de Catalunya, Spain
- Magyar Tudomanyos Akademia Szamitastechnikai es Automatizalasi Kutato Intezet, Hungary
The architecture is well thought out and I will continue to report on the progress of the research over the next year.