DATPROF

DATPROF Subset is used to create subsets of your existing production data for testing purposes. Subsets are generated using a single table as a base, but during the subsetting process you will need to

Fig 1 A data model (top) in DATPROF Subset

classify the other tables in your system or systems based on their relationship to that table, which will determine whether, and to what extent, those tables are included in your subset. This classification is assisted by suggestions made by the product, and can ultimately be visualised as either a data or process model, as seen in Figure 1. Both models are very helpful for understanding the structure of your database and therefore how to conduct the subsetting process.

Once you have created your subset, the product provides several ways to copy it into your test database. Notably, it allows you to append new test data onto existing tables that may or may not be populated. This is an enormous time-saver: you don’t need to regenerate all of your tables and re-copy all of your data whenever you add a new test case. Additionally, Subset takes care of duplicate data for you (or not, as appropriate). Options for recreating or refilling existing tables are also available. Subsets can also be deployed via DATPROF Runtime.

DATPROF Privacy is used to mask sensitive (test) data by applying one or more masking rules to it. It supports Oracle, SQL Server, PostgreSQL and IBM DB2 databases natively (with support for more forthcoming), and can (in theory) support any data source via a processing engine (which is to say, one of the aforementioned databases). It can also mask data located inside files that have been stored in a variety of formats, including CSV and XML. In addition, it masks live data directly inside each data source, meaning there is never any need to extract data (only metadata).

The product provides support for several masking rules out of the box, as well as custom masking rules and constraints. It allows you to enforce the order in which masking rules take effect by setting dependencies, masks consistently over all of your systems and applications, and maintains its own audit log, which is exposed as an HTML report after each run. As with DATPROF Subset, you can run the masking process via the product itself or by using DATPROF Runtime. It’s also worth noting that both DATPROF Privacy and DATPROF Subset complete their deployment operations by generating and running a SQL script. This ensures that they remain performant.

DATPROF Analyze is a data profiling and discovery product. Its chief relevance to test data management is in its ability to find sensitive data and personally identifiable information (PII) within either your subsets or your overall system. As a data profiling tool, it can also be used to understand how your data is being used, and hence inform the creation of subsets and synthetic data, as well as to investigate and identify data quality issues. It includes country specific profiles, allowing you to search for different varieties of PII depending on the country in which your data originates, and the profiling results it generates can be exported as an HTML report.

Fig 2 DATPROF Runtime

DATPROF Runtime (as seen in Figure 2) allows you to centrally manage your test data environments, testing teams and applications. It can be used to initiate and monitor a variety of test data management processes (such as subsetting and masking), comes with built-in error handling and logging (including a complete run history), and exposes a REST API for integration with, among other things, continuous testing platforms. It also natively integrates with both Tricentis Tosca and Parasoft.

Moreover, DATPROF Runtime is designed to facilitate ‘test data provisioning’, or the operationalisation of test data. It does this in two ways. Firstly, it enables self-service access to test data. Secondly, it allows you to proactively provide your testers with data. This is accomplished by generating application templates from your test datasets, which can then be deployed to any number of different machines, allowing testers to access the instantiated application using specifically the test data you have provided for them.

Finally, DATPROF Runtime also provides synthetic data generation capabilities, primarily as a means to fill in missing data in your existing test datasets. Although creating a synthetic dataset from scratch is possible, it is not a recommended approach except when you have no access to production data. A variety of generators are supplied, several of which are country specific. You can also opt to create your own.

The area where DATPROF most excels is ease of use: the various DATPROF products, although technically separate tools, feel like a single product that is exceptionally easy and intuitive to work with, and doesn’t require any significant training. This means that you can discover, subset, mask and deliver your data easily and – more importantly – quickly.

This is significantly enhanced by the most recent addition to the DATPROF product suite, DATPROF Runtime. Being able to create test data easily is important, but ultimately somewhat futile without the means to efficiently distribute that data to your testers. This is exactly what DATPROF Runtime provides. Moreover, by supplying a centralised location from which to manage your various testing assets and processes, it makes DATPROF easier to use than ever.

The Bottom Line

When we looked at DATPROF previously, we concluded that it was an exceptionally easy to use product with good – but not outstanding – functionality. In the time since, DATPROF has introduced data profiling, synthetic test data, and test data provisioning, among other things. In doing so, DATPROF has expanded on the latter without sacrificing – in fact, enhancing – the former.