Solix Technologies

Fig 2 - Data fabric

Enterprise data archiving is one of the four main areas of functionality, and Solix manages data pipelines to lower cost storage based on data retention rules set up within its core catalogue. The security and compliance offering includes areas such as data masking, and the identification of sensitive data, so that enterprises can be confident about their compliance with regulations. The Solix product has support for data lineage, so provides traceability.

It is the data lake functionality that is the most relevant for data fabric. Solix has search capabilities, both for structured and unstructured data, and these can be applied to any source that it is connected to. However in practice it can be difficult to deal with ad hoc distributed queries across a range of databases, particularly as such queries could interfere with the application performance of production systems. Hence most of their customers use Solix to bring data into a separate data lake, which itself can use either a SQL database (PostgresSQL compliant) or a NOSQL database like MongoDB, or native cloud storage such as S3 on AWS. In late 2023 Solix introduced an enterprise AI offering to early customers, allowing them to manage AI models from popular vendors like OpenAI and META, and to train such models on their own data lake data. This allows queries through natural language prompts as well as data visualisation and even sentiment analysis.

Solix offers a quite comprehensive solution to several aspects of data management. The product has a core data catalogue with significant data governance capabilities. It can connect to a wide range of data sources, from conventional databases like MySQL to applications like SAP HANA, NoSQL databases like MongoDB and even VSAM files, as well as streaming sources. Its wide range of customer deployments demonstrate that it operates at a considerable scale.  It is flexible in deployment, with multi-cloud support as well as the ability to run in a tightly controlled private cloud environment. The product has search capability across both structured and unstructured data, as well as links to various market-leading analytic tools to supplement its own offerings. The product has data preparation functionality, with hundreds of transformation and data quality checks built in, so Solix can be used as a data staging area, either to its own query engine or as a basis for loading into a separate data warehouse like Snowflake.

The bottom line

Solix has a broad range of functionality deployed at a wide range of customers. It has the core functionality needed for a data fabric architecture, though in practice most implementations are done via a data lake, which is a different approach to some other vendors. There are some definite advantages to this approach, with queries in the Solix platform not adversely affecting performance in the source systems. Provided that you are comfortable with this approach then Solix should be seriously considered, as its customer base is impressive and growing at a rapid clip.

Of the four solutions provided within SOLIXCloud, Enterprise Archiving is perhaps the most self-explanatory, offering a range of in-cloud data archival and application retirement features. Enterprise Data Lake enables the creation of a curated, centrally managed, “federated” data lake that in turn enables the centralised creation and orchestration of geographically or departmentally distributed data fabrics. Enterprise Content Services offers a centralised, automated, and API-driven cloud platform for enabling a variety of content-driven use-cases and lifecycle management of all unstructured data. And last, but certainly not least, Consumer Data Privacy provides several important data privacy and consumer data protection features, including data discovery tailored to finding all of a particular user’s data, the generation of data access reports, and the removal or redaction of sensitive data from your system. Note that much of the functionality on offer across SOLIXCloud is especially relevant – and overall, extremely helpful – when grappling with compliance mandates such as GDPR. Most of all, Enterprise Archiving includes robust data retention capabilities, and together with Consumer Data Privacy directly addresses both DSARs (Data Subject Access Requests) and GDPR’s ‘right to be forgotten’.

SOLIXCloud is more than just these four solutions, however. Considering Figure 1, we see that they are surrounded by ingestion, management and access layers. The former, Solix Connect, supports end to end connectivity across hundreds of data sources, while the latter, Solix Search, offers universal and self-service search access to your data that can be integrated with a handful of third-party products via API, most notably including Tableau, Qlik, and PowerBI.

Fig 02 - SOLIXCloud data lineage

The management layer, consisting of centralised metadata management and data governance, is provided via a data catalogue, which features a business glossary, policy management, policy-based data classification, and automated tagging on ingestion, among other things. The catalogue can also ingest data from other catalogues (although at present this does not go both ways: you cannot push changes made in SOLIXCloud to other products). Graphical data lineage is provided (see Figure 2), as are data quality and data processing. Data governance is catered to more specifically via the automated discovery, tagging and protection of sensitive data; data masking; and role-based access control, to name only a few relevant features. This ties into Consumer Data Privacy and the data retention management offered by Enterprise Archiving to create a wide-ranging compliance solution.

Fig 03 - SOLIXCloud purge configuration

For starters, SOLIXCloud provides access to much of Solix CDP’s functionality in a cloud-native setting. This is valuable in and of itself, but as with Solix CDP, SOLIXCloud is most prominently ahead of the pack when it comes to enabling regulatory compliance generally, via data retention in particular. The presence of Consumer Data Privacy and Enterprise Archiving alongside its more standard data governance functionality is a significant factor here. In particular, it allows you to set data retention policies that effectively automate your retention management, up to and including removing expired data automatically. Structured data can be purged systematically (as shown in Figure 3), such that your system as a whole always remains consistent and referentially intact, while unstructured data can be redacted (in other words, blacked out) from whichever document it’s embedded in. You cannot currently purge documents containing sensitive data in their entirety (according to Solix this will be present in a future release) but even without that capability, SOLIXCloud is substantially ahead of the curve when it comes to data retention.

Unfortunately, SOLIXCloud suffers from somewhat limited third-party integration. For example, it will pull from (but not push to) other catalogues (though this is planned for a future release), and it can’t ingest data lineage from other sources. That said, the platform is highly integrated within itself (which is to say that it is interoperable) and the selection of data sources available through Solix Connect is impressive. What’s more, we are told that much of these integration concerns will be addressed (if not, remedied entirely) in the product’s next release.

The Bottom Line

SOLIXCloud provides many of the capabilities of Solix Common Data Platform in a cloud-native format. In doing so, it offers a cloud data management solution that is particularly adept at enabling regulatory compliance via sophisticated data archival and retention management.

Fig 01 - Solix Common Data Platform architecture

The core of the Syniti Knowledge Platform is its knowledge graph, which contains both business and technical assets. In turn, your graph and its assets are accessed via the platform’s data catalogue. This allows you to view, edit and manage all of your assets, including strategic assets such as initiatives, strategies and goals; governance assets such as rules, policies and business terms; and, of course, data assets. Search access to these assets is provided (as shown in Figure 1), and all of them include a plethora of associated metadata, including related assets. For example, this could list associated business terms or applicable rules and policies. It could also contain business goals or strategies that the asset is contributing towards, or a business representation of the asset if it is primarily technical. In the latter case, the intention is that technical assets are linked directly to business assets that demonstrate what each asset does and why it is important.

Moreover, the assets within Syniti are democratised. All users are able to share their own descriptions or definitions for any business asset, and to comment on or endorse anyone else’s. In turn, each asset is equipped with a list of subject matter experts, or ‘sponsors’, who are responsible for curating their assigned assets, using these crowd-sourced suggestions and endorsements as a guide. When a user requests a change to an asset, the platform initiates an automated workflow that polls each of that asset’s sponsors for their opinion on that particular change. When and if a change wins majority approval, it can be implemented automatically.

What’s more, the Syniti Knowledge Platform understands assets within their business context. This consists of both identifying the relationships and connections between your assets and leveraging those connections to create business value, as evident within the product’s DeepGuidance capability, which uses machine learning to automatically and dynamically suggest improvements to your assets as you are viewing them. This could include suggestions for new business rules, acknowledgement and remediation of data quality issues, or detection of suspected – but undocumented – asset relationships, to name only a few examples.

Fig 02 - Purge Configuration

For business terms in particular, natural language processing is employed to detect and highlight words or phrases that may represent additional, as yet undefined business terms. This is seen in Figure 2, along with several DeepGuidance suggestions. In turn, these terms can be referenced within business rules, which are written in natural language using business terminology. A given rule will also contain its written implications (in other words, its business meaning) as metadata, as well as an enforcement profile that shows where and how it is being enforced (for example, a data quality rule might link to its entry in your data quality solution). Policies operate in much the same way as rules, but as you might expect are much higher level. More importantly, terms, rules, and policies often act as connective tissue between your technical and data assets and your business goals and strategies, enabling you to understand precisely how the former relate to the latter.

The Syniti Knowledge Platform has a number of properties that make it very suitable for data governance. It allows you to consolidate and curate all organisational knowledge within a single platform, coaxing knowledge out of any silos that have built up and making it accessible to all; it enables technical assets and knowledge to be tightly coupled with business context and strategy, allowing you to ensure that they are contributing positively to business outcomes; and it does all this while encouraging collaboration, reuse, and understanding.

The collaborative capabilities that Syniti provides are particularly worth noting. As mentioned, the business terms, policies, and other assets within the platform are crowd-sourced. This means that everyone within your organisation will have the opportunity to help define them. This is a good thing, both for allowing your organisation as a whole to share understanding and knowledge of what your assets mean and how they are used in practice, and for allowing the users of a given asset to define it in the most useful possible way. What’s more, the fact that Syniti is not licensed on a per user basis means that your entire organisation will be able to participate in this process.

Syniti’s DeepGuidance capability is also notable: the ability to make AI-driven dynamic suggestions and recommendations across your entire catalogue is very powerful, both for promoting understanding and for driving positive change across the platform.

The Bottom Line

The Syniti Knowledge Platform prioritises the business value and the democratisation of your data and knowledge assets above all else. If you share these priorities – and there is every reason to – it is absolutely worth your consideration.

Fig 02 - Discovery dashboard in Solix Common Data Platform

Sensitive data discovery in CDP allows you to identify and locate sensitive data across all of your applications, then catalogue it within a single platform. This not only supports regulatory compliance but provides additional value when used in conjunction with the data security and analytics features contained within CDP. For example, you could leverage CDP to discover your sensitive data, mask or encrypt it, then expose it to your analytics tools via an API. Equally, you could use it to find and protect the sensitive data hidden in your data lake, helping to prevent it from turning into a data swamp. The overall point is that CDP doesn’t just provide sensitive data discovery by its lonesome, but as the first step in addressing an end-to-end use case.

Fig 03 - Data flow diagram in Solix Common Data Platform

The discovery process itself examines both your data and the metadata attached to it. The process begins with the ‘low-hanging fruit’ of metadata matching (for instance, by examining column names) before moving on to the more difficult – but also more thorough – techniques of pattern and value matching within the data as well as master data lookup matching. Moreover, over 25 predefined classification policies are provided out of the box. These are based on compliance requirements such as GDPR, and have been designed to locate any PII, PCI or PHI data elements within your system. You are free to use these policies as-is, customise them, or create your own. The discovery process is fully configurable, and may include sampling to help your users detect false positives. Moreover, sensitive data is always discovered in place: it is never moved by the discovery process. Discovery dashboards and data flow diagrams (as shown in Figures 2 and 3) are provided to visualise your discovery results, and the process itself (as with many of the capabilities in CDP) is available to be run in the cloud as a service.

Solix also provides a dedicated GDPR tool for finding and actioning on all data that is sensitive under GDPR specifically. This will certainly include data discovery, but could also mean purging, archiving, or masking your sensitive data, depending on your requirements.

The sensitive data discovery capabilities offered by Solix are mature and competent. The out of the box policies provided allow you to hit the ground running and receive value very quickly, and this is particularly true in Oracle and PeopleSoft environments, for which dedicated project accelerators are provided, further reducing implementation time and improving ROI. Moreover, the dedicated GDPR tool the company offers provides a relatively easy way of getting your house in order for GDPR. Given that GDPR is a major driver for sensitive data management, this is a significant draw.

One of the other big benefits for CDP is that it is not by any means restricted to sensitive data discovery. On the contrary, it provides a broad range of (sensitive) data management and security capabilities, such as data masking, database archival and application retirement. These naturally pair well with sensitive data discovery. Moreover, CDP is intended as a single, scalable and all-encompassing solution. Not only does it provide a large quantity of data management and security features, it will also deploy those features across a wide range of environments and data sources, notably including big data environments such as data lakes. The only caveat to this, at least in terms of sensitive data discovery, is that its support for NoSQL data sources is relatively limited. That said, within the context of the sensitive data discovery space, it offers as much or more than many of its competitors.

The Bottom Line

Solix CDP offers sensitive data discovery as one feature among many within a broad and highly competent data management suite. Given that sensitive data discovery works most effectively in tandem with other data security and sensitive data management capabilities, this is no bad thing.

Fig 01 - Data subsetting using Solix Technologies

For data subsetting, two methods are supported. The conventional approach is to extract your subset against a defined parameter, which in this case can span your database either horizontally or vertically (meaning you could subset by table or, for example, time). Alternatively, you could clone your database before removing or truncating data in excess of your desired subset. This is helpful (and performant) if the latter is large relative to the overall size of your database, or if your data model is particularly complex. Both methods support self-service and maintain referential integrity, and they are each shown, albeit at a high level, in Figure 1. The platform also supports snapshot technology, provided by hardware vendors such as NetApp and EMC.

Fig 02 - Data masking using Solix Technologies

As far as data masking (see Figure 2) is concerned (which may be required to secure sensitive data in test subsets), Solix can mask consistently, within single or multiple databases, and without losing referential integrity or moving your data outside of its original data source. It offers a variety of masking rules and algorithms, some of which are generic (for example, a rule that produces a random sequence of letters) while others are specific to particular types of data, for instance date of birth or social security number. Custom algorithms are supported and can be written in Java, T-SQL or PL/SQL, as are masking algorithms for both structured and unstructured data (documents, in particular), the latter primarily via redaction.

A notable differentiator for Solix is that it offers pre-packaged masking capabilities for Oracle and PeopleSoft application environments. In addition, Solix provides masking support across most leading relational databases, as well as MongoDB and flat files, and has implemented its masking algorithms natively in Oracle PL/SQL.

Fig 03 - Mapping out your data landscape using Solix Technologies

The product’s masking capabilities are complemented by a data discovery functionality that can scan, and subsequently ‘map out’, your enterprise data landscape, including both production and non-production environments. The results of this process can be seen in Figure 3. This capability examines both metadata (such as column names) gathered during data ingestion and actual data (by looking for patterns in the data values). To reduce false positives, it uses sampling to help users decide what is and is not sensitive. The solution comes pre-populated with discovery rules suitable for identifying PII, PCI, PHI, and other such industry-defined sensitive data elements. It also provides a dedicated GDPR tool for finding and actioning on data that is sensitive under GDPR. This could mean identifying, purging, archiving, or masking, depending on the situation.

Test data generation is available, but is intended as a way to augment your subsetting and/or masking processes rather than as a standalone process. To that end, there is tooling provided to ensure that generated test data is representative of your real data, and there is an embedded rules engine to help automate the data generation process.

Solix is first and foremost a data management solution, and what’s more, one that boasts particularly strong data governance, regulatory compliance (particularly data retention), data privacy, and support for the cloud. TDM capabilities are delivered, and must be understood, in that context. Solutions for data masking, subsetting and discovery are all on offer, and are each mature and effective at what they do. That said, if you are licensing Solix purely for TDM, you are missing out.

There are two reasons for this. Firstly, and most importantly, Solix is much more than just TDM. The platform as a whole is sufficiently comprehensive and far-reaching that confining it to one corner of your environment is, quite frankly, a waste. The second reason is that, as far as TDM goes, Solix is certainly good, but it lacks some of the more advanced features (you could say bells and whistles) that might be available from a more dedicated product. The question you need to be asking is: are those features worth the cost, when Solix is perfectly adequate for TDM and offers so much more besides?

The Bottom Line

Solix provides a capable TDM solution that will be of particular interest if you are struggling with data privacy and/or regulatory compliance issues, and even more so if you need a general solution for enterprise data management that includes TDM.