DevOpsQA issues a 60-hours challenge - Some observations around, but not limited to, ISO 20022

ISO 20022 is a flexible and extendable International Standard to enhance the interoperability of business transactional information within and between financial institutions and their customers. It has been prepared by ISO Technical Committee TC68 Financial Services. According to this, it describes a common platform for the development of messages using:

• a modelling methodology to capture in a syntax-independent way financial business information, required to execute business transactions and associated message flows.
• a central dictionary of business concepts, components and rules used in financial communications.
• a set of data points of interest and the associated rules required to execute a business transaction abstracted from the underlying technical syntax/ format of interface and transport protocols, i.e XML and ASN.1 design rules to convert the message models into XML or ASN.1 schemas, whenever the use of the ISO 20022 XML or ASN.1-based syntax is preferred.

The resulting models and derived messages are published in a shared knowledge base and the Catalogue of messages and associated usage rules are stored in the ISO 20022 Data Dictionary Repository. This flexible framework allows communities of users and message development organisations to define message sets according to an internationally agreed approach using internationally agreed business semantics and, whenever desirable, to migrate to the use of a common XML or ASN.1-based syntax.

Here’s a video explaining the important bits.

The ISO 20022 standard seems to me to be an enabler for reliable collaboration and communication, quite apart from any regulatory requirements associated with it. You’ll want to comply in the interests of communication, reliability, and resilience. Standardising communication via a data dictionary not only makes building systems easier; it also makes it more likely that a lack of understanding will break something visibly before it reaches production. I am always surprised that organisation-wide data dictionaries are not more widely used, perhaps ISO 20022 will be a catalyst for further adoption.

Now, a part of adopting ISO 20022 is preparing test data packs and running cost-effective testing of adherence. I’ve just attended a webinar with Harry Burn (Senior Engineer/Solution Architect) of Curiosity Software and Derrick Beling (founder and CEO) Marcus Henry (Enterprise Architect & Management Consultant) of South African consultancy DevOpsQA. You can view the webinar here, and there is an ISO 20022 deadline (November 2025 when SWIFT is scheduled to retire its existing MT message standard for payments) for developers of (especially) banking systems to be aware of.

I’m not going to go through the webinar slide by slide. It’s referenced above and you can listen to it as well as I can. But I will pick out some key points.

Derrick sees end-to-end regression testing as too slow and inefficient and was attracted by Curiosity Software’s “3-minute Test Automation Challenge”, taking advantage of its Test Modeller’s built-in automation frameworks and Web UI scanner. He claims that he can effectively manage testing quality and the quality of an ISO 20022 implementation in under 60 hours.

Derrick identifies problems worth solving:

• Production bugs;
• Time to remediate bugs;
• Slow test data provisioning;
• Incomplete test data (missing edge conditions);
• Test data for data truncation (when you only want part of a message or an arriving message is too long);
• Insufficient automation and skills;
• Inefficient regression and end-to-end testing; and,
• Waiting for code delivery.

As an aside, I’d rather talk about “defects” than “bugs”: bugs are simple coding errors and the like, with comparatively small scope of impact; “defects” might include a system that embodies a gross misunderstanding of the way the business operates. Still, it is a good list to address.

Derrick starts by dividing up the problem into data flows, business transactions, technical transactions and extra data needed for compliance. This helps, but banking systems are inherently complex and need a great variety of tests – and the quality of tests matters (at minimum, each test must have a good chance of finding a defect). Just using (sanitised) production data doesn’t cut it (unless you get lucky). It is better, more efficient, to design a test pack that, say, actively targets boundary cases (test the boundary, and 1 above and one below).

That could be a lot of tests, so automation is your only practical option – but you still need to design the testing to find defects. Tests that don’t find defects may give you a warm feeling, that everything is right with the world, but they are essentially a waste of resources, a missed opportunity to find a defect.

Then Derrick goes into ISO 20022 in some detail. It looks to me to be a reasonably useful standard, quite apart from being a SWIFT requirement soon. He describes the ISO 20022 recipe and the place of the Data Dictionary in it; and how to apply it. There are several demos in the webinar, and it is well worth watching, in my opinion.

I also found a quick Hints and Tips checklist for migration to ISO 20022, which makes a lot of sense to me, on the Curiosity Software website here.

I do have some potential concerns about companies “going for” ISO 20022 compliance. For a start, is there an official certification authority? If not, does “compliance” mean much? In the absence of official certification, perhaps “best endeavours” is about as much as one can hope for. This is what ISO says: “There is no official certification authority for ISO 20022, and the implementation of ISO 20022 message definitions will depend a lot on the specific requirements of the community that is implementing. The ISO 20022 Registration Authority (RA) and Technical Support Group (TSG) have defined an ISO 20022 Compliance Checklist providing guidance to implementers about some key aspects to be considered in order to be as compliant as possible with the standard. The checklist can be used by implementers, adopters (consultants, tool providers, service providers, etc.), and consumers of ISO 20022 messages to tick whether they have considered each of the key aspects related to ISO 20022 compliance”.

Of course, if your messages don’t comply, your transactions will simply fail following the ISO cut-off dates, as your messages will not be accepted by other organisations’ payment systems (or by SWIFT). But in my experience interpretations of standards can differ in subtle but important ways, and it is good to discover this before going live – and certification helps.

I’m also a bit worried that some companies will worry more about the letter than the spirit of ISO 20022. Probably not, if DevOpsQA is advising them, but not all companies take expert advice.

Another question I might ask is: “Is it possible to exhaustively test ISO 20022 compliance and yet still have a defect in the overall process?” I imagine so, even if the discipline of ISO 20022 helps developers to make fewer mistakes. If so, how does the business assess whether it has done “enough” testing, beyond ISO 20022? Won’t some people see mere ISO 20022 compliance as “good enough” – so that is the only testing (beyond Dev unit testing) that they do?

Choosing completion criteria for testing is far from trivial, as Glenford Myers pointed out in the last century, in his book “The Art of Software Testing”. Defensible completion criteria include “we’ve stopped finding defects” (so further testing, with that test pack at least, is a waste of money); “we have found about the expected number of defects in this amount of code” (yes, there are estimates of defects per line of code available); “we think that the risk of failure has been reduced to the point that not releasing this product to the business puts company profits at more risk than that associated with any likely production issues” (you will need to show your working). This is a huge topic, one which we have no room for here, but I do think that, however useful ISO 20022 adherence is, you must always think about defect removal holistically, and in terms of risk to the business.

You will get no brownie points for ISO 20022 adherence, if the real content of your nicely formatted communications is often wrong. Validating ISO 20022 adherence is necessary, but not sufficient.