putilov_denis - stock.adobe.com

Xandr's bumpy ride to AI software test automation success

AI-based software testing tool Mabl reduced bugs and testing complexity for advertising company Xandr, but the initial implementation process was far from smooth.

After an awkward start, a Microsoft-owned advertising exchange company streamlined software testing with AI-assisted software test automation tool Mabl.

Xandr, which connects buyers and sellers of digital and television advertising space, had struggled with testing because of its complex microservices architecture and ephemeral testing environment. The company installed Mabl, a software test automation tool that uses AI to recommend and learn from fixes, in February 2021.

This type of AI-assisted software testing is a relatively recent concept, and its widespread adoption has been stymied by issues such as gaining developers' trust. But while the Xandr team experienced a few initial bumps in the road, the successful implementation of Mabl resulted in a simplified testing environment that staff other than highly trained software engineers can use, according to Jesse Silverstein, senior engineering manager at Xandr.

"[Mabl] has taken away a significant amount of complexity around testing," Silverstein said. "It has opened up the ability for more people to test, for the same reason."

Low-code Mabl a software test automation standout

Although Xandr presents one UI to clients, a dozen UI microservices owned by different teams process millions of requests per day behind the scenes. Silverstein's initial attempt at UI testing used an in-house testing framework built with WebDriver-based Node.js Selenium tools.

You can add assertions every step along the way, and it makes writing the test significantly easier than any other product we explored. It's certainly easier than the things that I had built.
Jesse SilversteinSenior engineering manager, Xandr

"It was sufficient for highly motivated engineers, but the barrier to entry was too high," he said. "You needed to learn too much about the testing framework, the way to use the framework, and the capabilities and shortcomings of the testing framework in order to get started writing tests."

Rather than build another testing framework from scratch, Silverstein began to look for a software test automation tool that might fit the bill. His team considered AI-assisted tools Functionize and Appvance, but decided on Mabl because of its low-code approach.

"You pull up your product, click 'record' in the trainer and then just use the product, and it records everything that you did," he said. "You can add assertions every step along the way, and it makes writing the test significantly easier than any other product we explored. It's certainly easier than the things that I had built."

Early struggles with Mabl

Initially, Mabl's secure network tunneling mechanism was incompatible with the naming conventions one Xandr team used for Kubernetes deployments. When Mabl triggered the performance testing apparatus for continuous integration, the identifier it provided would correspond to the most recently created ephemeral Kubernetes resource, which would override the intended environment target.

Jesse Silverstein, senior engineering manager, XandrJesse Silverstein

After adding a test to the plan, the team sometimes had trouble getting the trainer to target a different ephemeral environment as well, Silverstein said.

Mabl's engineering team addressed these problems in less than a week, according to Silverstein. "They built a whole new interface for how you launch the trainer," he said. "It's now working perfectly."

Xandr's team had been launching Mabl environments into Azure Kubernetes, creating a separate Kubernetes cluster for each of those instances. Mabl provided a Docker container image that opened a secure tunnel from wherever the container was deployed to Mabl's cloud for testing, but that connection was unreliable.

"If we tried to [give it] the same name as an ephemeral environment, it wouldn't connect," Silverstein said.

While earlier versions of the software test automation tool did not support the reuse of environment names for Mabl links, that limitation has since been removed and all customers can now reuse environment names, a Mabl spokesperson said.

Mabl's quick response to Xandr's implementation issues impressed Silverstein.

"They partnered with us closely and figured out all the things that were hard to do -- and in some cases, impossible -- and immediately got to work," he said.

Dig Deeper on Software testing tools and techniques

Cloud Computing
App Architecture