Testim

Fig 1 - Editing a test in Testim Automate

Automate creates your tests as you use your application by using a Google Chrome extension to capture your actions as a series of test steps. These can then be configured into an automated test using Automate’s low-code visual editor (see Figure 1). This could include grouping steps together (forming a reusable test component), or adding validation, loops, parameterisation, and so on. You can further augment your tests with steps that leverage custom JavaScript and/or API calls, and to this end the platform comes with a full-fledged JavaScript editor, as shown in Figure 2.

Fig 2 - Creating a JavaScript step in Testim Automate

Automate distinguishes itself by automatically creating ‘smart locators’ for every element used in your test. These use AI to intelligently locate said elements at runtime for your tests to operate on, regardless of any stylistic or structural changes that have occurred in the meantime. While executing the test, each element’s properties and surrounding context are examined to find the matching element. Every part of any given smart locator, including its minimum confidence level, is fully transparent and can be adjusted if necessary. In effect, smart locators allow your tests to self-heal in the face of superficial changes, increasing resiliency and substantially reducing the need for maintenance.

A second major AI capability is the ability to automatically find steps that have been duplicated over multiple tests, then group them into reusable test components and refactor existing tests to accommodate them. This includes steps that have the same structure but use different data, in which case they will be automatically parameterised. This capability also helps prevent the creation of duplicate steps during test creation, by detecting patterns and suggesting existing groups to auto-complete the test case.

Automate also provides simple and effective root cause analysis: when a test fails, the platform will present screenshots for each test step up until the point where it failed, alongside the relevant console log, network activity, and ‘baseline’ screenshots from a previous passing test. Additionally, failed tests are grouped (and can be analysed) by failure type, and errors can be presented in aggregate to save time triaging and prioritising work. Testim also suggests the failure cause and can learn from previously tagged failures to help identify recurring issues.

In addition to all of the above, Automate comes with a significant TestOps capability. This includes full source control for your test assets, including branching, merging, pull requests, and conflict resolution; test management, via a centralised, searchable test library; and test insights, such as failure type and reporting trends. Notably, tests can be equipped with a test owner, and always come with a test status, which can be draft, active, quarantine, or ‘evaluating’. The latter is the most interesting, as it means the test will be deployed as part of your CI pipeline, but that its failure will not interrupt the build. As its name suggests, this is useful for evaluating tests before deploying them in full.

Testim Automate provides several notable benefits, over and above what is typical for a test automation product:

• Tests are easy and fast to create.
• Grouped test steps enable reuse, and automated group discovery both reuses and refactors, reducing unnecessary duplication and driving collaboration.
• Smart locators make your tests far more robust, and can quite dramatically reduce time spent on maintenance.

Moreover, when your tests fail, you will be able to trust that it is because something has actually gone wrong with your system, not with your tests themselves. This should make implementing testing into your CI build processes far more palatable and efficient, by making it much less likely that your build will fail due to a need for test maintenance.

There are also other, perhaps less prominent, benefits. For instance, Testim caters well to both technical and non-technical users. It also emphasises TestOps, and with good reason: TestOps helps your testing to scale up (and shift left) in order to keep pace with ever-accelerating development pipelines, and thereby prevent your tests from becoming a bottleneck to your development.

The only aspect we care to criticise is the product’s relatively small amount of high-level design automation, usually exemplified by model-based testing or integration with the same. That said, the keyword here is ‘relative’: Testim already provides a number of the benefits of model-based design, including enabling improved test resilience, step reuse, and collaboration via its grouping and smart locator features, and it does so without the significant onboarding difficulties that tend to come with more fundamentally model-based approaches. Even so, we would be pleased to see this capability enter the picture in some fashion, perhaps through partnership with existing solutions.

The Bottom Line

Testim Automate uses AI to help you build automated, robust, reliable, CI-ready, and ultimately trustworthy tests. It creates these tests quickly and with superb ease of use, while helping you manage and scale up your testing operation. If you want to revamp and accelerate your testing process, it is certainly a product worthy of your attention.