Analyst Coverage: Daniel Howard
Eggplant was founded in 2008 with the goal of helping organisations put users at the centre of software testing. In 2013, Eggplant developed a performance testing capability, and in 2017 it launched Eggplant AI, an intelligent, AI powered test design automation product and part of the Digital Automation Intelligence suite. In 2018, it acquired the Web Performance division of NCC Group to further expand its capabilities.
Eggplant is formally based in London, but in reality it is a global company, with several international offices, a long list of business partners, and over 400 active customers.
Last Updated: 26th April 2018
For Eggplant, testing and testing software should, above all else, increase customer satisfaction and improve the experience of the end user. In their words, testing should result in a product that “delights” its users. Unfortunately, this is not often the case. According to Eggplant’s research, the vast majority of companies which employ testing are regularly meeting their testing objectives, yet only a small fraction considers themselves to be meeting customer expectations. Not only that, but on average, almost a third of IT budgets are spent on testing and QA, yet close to 90 percent of QA teams cannot keep up with their workload.
Eggplant addresses both of these issues with its Digital Automation Intelligence Suite, and particularly with Eggplant AI, a product within that suite which uses artificial intelligence, machine learning, and analytics to find defects and maximise coverage of user journeys. Eggplant AI improves efficiency and productivity by providing automation in every area of testing, not just test execution. In addition, to ensure a good end-user experience, everything within Eggplant AI is done from the perspective of an end user. For instance, testing is driven through the user interface (UI), and emphatically not the backend or the application code. Moreover, testing is not restricted to user journeys that appear sensible, taking the more realistic approach that any and every path through the application is a valid one.
Eggplant has a long list of business partners, including resellers such as CGI Group, Pactera, and IT Ecology, that resell the product and have access to the support, services, and technical training needed to help deliver it; service providers, such as NCC Group, Sogeti, Cigniti, and Infosys; and systems integrators, such as IBM, Tata, and Capgemini. Both of the latter groups work with customers to deliver comprehensive test automation solutions.
Between their offices and their partners, Eggplant has well over 400 active customers in upwards of 30 countries, and have served more than 600 customers in total since their inception. These customers come from all manner of sectors, including automotive, aerospace, defence, financial services, healthcare, media, and retail.
Eggplant AI uses model-based testing to automate test case generation. This means that you create a graphical model of the system under test that is then leveraged by the product to automatically generate test cases and related assets. Furthermore, the product uses machine learning and predictive analytics to ensure that all new test cases will either increase test coverage or retest a path that has previously resulted in a defect. In addition, Eggplant AI is able to learn patterns from previous tests that have resulted in errors, and will then repeat these patterns in new test cases until the errors stop occurring. Eggplant AI also comes with comprehensive reporting capabilities, including the run report, which details test execution status; the coverage report, which lists each object in your model along with their test coverage percent and may additionally be displayed graphically as a heat map of your system model; and the test case report, which allows you to search your run report for a specific set of test cases.
Apart from Eggplant AI, there are several other products present in the Digital Automation Intelligence Suite. Eggplant Functional tests from the user perspective through intelligent image and text understanding, API automation, and WebDriver object automation — all within a single test. Eggplant Performance and Eggplant Network together form a performance testing solution capable of testing in a variety of network and performance conditions on demand. Eggplant Automation Cloud allows you to generate and administer a centralised, cloud-based testing lab. Eggplant Manager helps you to manage your test execution process on a large scale, complete with continuous integration.
Eggplant offer FastStart services to shorten implementation times and get teams up and running faster. The company also provides training and certification in a number of Eggplant products, extensive documentation (available in English, Chinese, and Japanese), and a community forum. In addition, all of their partners are given access to the latest tools, training, and other support needed to deliver a full-fledged test automation solution.
Last Updated: 5th February 2020
Mutable Award: Gold 2019
The Eggplant Digital Automation Intelligence suite aims to automate every aspect of your product quality pipeline. In Eggplant AI’s case, AI and machine learning are used to automatically generate maximally covering test cases and actively target defects. In addition, to ensure your testing results in a good end-user experience, everything within Eggplant AI is done primarily from the perspective of an end user. Moreover, testing is not restricted to user journeys that appear sensible, taking the more realistic approach that any and every path through the application is a valid one, thus mimicking the real-world behaviour of your users.
Apart from Eggplant AI, the Eggplant Digital Automation Intelligence suite also contains Eggplant Functional, for test execution, and Eggplant Release Insights, which provides metrics for measuring and analysing release quality, including predictors for upcoming releases. Outside the suite, Eggplant provides performance and load testing as well as various customer monitoring products, including an entire second suite of products – Eggplant Customer Experience Insights – dedicated to monitoring your application once it’s in production. The two suites are fully compatible, and are available together as Eggplant Customer Experience Optimization.
“It’s so important to test the whole product, and testing it through the UI is the best way to do that. By running and looking at an Eggplant script, you can tell what the user was doing, what the business logic was, and what the UX was intended to be... So far, I haven’t found anything that Eggplant can’t do.”
“Other tools focus on testing at the code level, but Eggplant allows us to test our systems like a user.”
“Eggplant provides the best overall mix of capabilities from an end-to-end test perspective.”
Eggplant AI uses model-based testing to graphically model your system under test, as shown in Figure 1. You can create your model manually, import an existing model, or generate your model automatically from Eggplant tests, Gherkin feature files, monitoring data, or other sources. This process is intelligent, and will detect all valid paths implicit in the imported files even if they are not defined explicitly, thus unlocking additional coverage from your existing assets without any additional effort.
Your model can then be leveraged to automatically generate test cases, using AI to maximise the increase in test coverage that each new test case provides. You can create test cases manually by selecting a path through your model, import existing test cases from, say, Selenium, and weight individual parts of your model to make them proportionally more or less likely to be used in your automatically generated tests. Furthermore, the product uses machine learning to identify and understand patterns in your tests. This includes risk patterns, which can be leveraged alongside monitoring information to ensure that the most critical or valuable tests are always executed first, as well as any common patterns in your failing tests, which will be repeated in new tests until they stop resulting in failures, at which point they will gradually (and automatically) be unlearnt.
Your generated test cases will be executable if the corresponding parts of your model have been equipped with code snippets, which can be written manually or generated automatically using Eggplant’s ‘autosnippets’ image capture functionality. In addition, an accelerator allows you to generate your model directly from these autosnippets. Eggplant’s Universal Fusion Engine allows you to execute your scripts against any system, browser or device non-intrusively and without modifying any of your testing assets, and you have several ways of viewing your test results, including a coverage heat map as seen in Figure 2.
Eggplant also provides performance testing via Eggplant Performance, which is highly integrated with Eggplant AI. To wit, any Eggplant AI model can be used to create performance tests automatically. This goes a long way in making performance testing more accessible.
For Eggplant, testing should, above all else, increase customer satisfaction and improve the experience of the end user. In their words, testing should result in a product that “delights” its users. Unfortunately, this is rarely the case. According to Eggplant’s research, the vast majority of companies which employ testing are regularly meeting their testing objectives, yet only a small fraction considers themselves to be meeting customer expectations. Not only that, but on average, almost a third of IT budgets are spent on testing and QA, yet close to 90 percent of QA teams cannot keep up with their workload. In short, current testing is both insufficient and laborious. There is only one plausible solution: automation, and lots of it.
Accordingly, the automation present in Eggplant AI is extremely extensive, to the point that even test design is handled primarily via AI. This automation is suitable for either replacing or augmenting your more manual efforts, but either way it promises to dramatically increase the speed at which you can produce test assets, while at the same time maximising coverage and minimising bias. Moreover, the product provides several highly automated methods for generating your initial model, whether you have a suite of existing test assets or not. This makes it exceedingly fast and easy to get up and running with an Eggplant AI model, and by extension Eggplant AI.
The other products Eggplant offers are also highly valuable. Performance testing is a key component for ensuring a good customer experience – poorly performing apps are rarely well received – and the sophisticated and extensive release and customer monitoring Eggplant provides is highly useful for shifting right, managing change, and ensuring that your applications continuously improve over their entire lifetime. In combination, these products allow Eggplant to deliver a vast swathe of testing capability within a single platform via an intuitive, uniform experience. Furthermore, this intuitiveness allows your existing nontechnical users easy access to your testing process, empowering them to play an active role in it.
The Bottom Line
Eggplant is laser focused on enabling you to provide an exceptional customer experience via highly efficient and automated testing backed up by extensive analytics. Eggplant AI is both a core part of the company’s offering and a leading product in the Test Design Automation space. It should not be overlooked.
Mutable Award: Gold 2019