Eggplant Generative AI

With the recent boom in interest in generative AI (GAI), Keysight has been quick to incorporate it into its Eggplant test suite, creating what it sometimes refers to as AI-augmented testing.

This has led to the company expanding the scope of its test design automation efforts. Specifically, Eggplant now allows you to ingest various sources of knowledge, most prominently requirements, in either structured or unstructured data formats. The knowledge therein can then be extracted and recombined into a structured format that is easier to operate on (although we should note that this process is non-trivial, and can even be quite complex). Eggplant subsequently leverages GAI on the newly structured data to automatically create various test assets from it. This includes the test scripts and digital twin models that are the hallmark of Eggplant’s testing apparatus. Since these assets are generated automatically, they can also be regenerated automatically whenever your requirements change (as they so often do). This can extend to a proactive, delta-based analysis (including risk assessment) of any new and/or updated requirements, followed by a corresponding refresh of the related test assets. This should serve to add resilience to your testing suite.

From an architectural point of view, the core this of GAI technology is, of course, the Large Language Model (LLM). In this case, an LLM is used to drive all of this automated generation. At the same time, Eggplant is not tethered to any specific LLM, and in fact is designed so that you can swap out the LLM that it uses at more or less any time. This means that you can, for instance, leverage it immediately using an off-the-shelf LLM (ChatGPT, say) then switch to a more customised, homegrown LLM if one becomes available. Eggplant also uses a Retrieval-Augmented Generation (RAG) architecture, although that is already becoming fairly typical for GAI-based technologies.

We should also note that the use of GAI as part of Eggplant is done with considerable care. In particular, there are substantial efforts taken by the product to properly comprehend your requirements before it feeds them to an LLM. This is accomplished using both Natural Language Processing (NLP) and its subfield Natural Language Understanding (NLU). To wit, the product will automatically create a knowledge graph based on your requirements, which will in turn fuel a graph-based neural network to assist in understanding them. Combined with a data dictionary, this allows Eggplant to understand both the context and the language of your requirements (and other knowledge sources, for that matter). This enables it to resolve ambiguities in your requirements, detect when a linked document isn’t available, request clarifying information, and so on. In the future, Keysight intends to extend these techniques even further by incorporating knowledge of industry-specific terminologies and practices.

In short, Keysight has made an effective first attempt at capitalising on GAI technology. We look forward to seeing how this attempt develops as both the solution itself and the technology underpinning it mature.