Event-Driven Ansible early adopters share lessons learned
Beta testers of the new Event-Driven Ansible tool from Red Hat said expanding automation and moving toward self-healing systems requires fresh skills and organizational shifts.
BOSTON -- IT pros that beta tested Red Hat's Event-Driven Ansible plan to make it part of expanding their company's automation practices but said that expansion will require much more than a new tool.
The middleware team at Madrid-based multi-national insurance company Mutua Madrileña created its own event-driven architecture for Ansible two years before the feature became an official part of Ansible Automation Platform. In both the homegrown system and the Event-Driven Ansible product, alerts from these tools automatically kick off Ansible automation workflows, an architecture designed to make systems self-healing instead of requiring a manual response from IT.
Mutua Madrileña's middleware team wrote its own webhooks to link Ansible with Dynatrace, Prometheus and Elastic observability tools when its five members were tasked with managing 60-plus application platforms -- more than double what it had when it first started using Ansible in 2016.
Event-driven workflows that restart application servers, compress log files and increase resource quotas in response to outages and capacity constraints were the only way out of an overwhelming situation, said Marta Ceciliano, head of the Mutua Madrileña middleware team, in a presentation Wednesday at the Red Hat Summit.
"The lifecycle for every project in middleware had nine stages" -- most of which, including provisioning, security and incident response, were manual before the event-driven architecture effort began in 2021, Ceciliano said. "Multiply those nine stages by 60 platforms, and you get a mess."
Before the event-driven workflows, basic IT issues could take up to two hours to resolve. Now with predictive observability tools from Dynatrace triggering workflows in Ansible, "we can get to a no-outage situation," she said.
Ansible automation meets organizational friction
Mutua Madrileña automation lead Yamandu Tellechea said he expects that Event-Driven Ansible as an officially supported feature will act as a centralized hub for multiple event sources that can be combined into more complex workflows rather than individually linking event sources to Ansible from the outside in. Pre-packaged integrations that will accompany Event-Driven Ansible's general availability with Ansible Automation Platform 2.4 next month, include Cisco ThousandEyes, CyberArk, F5, IBM Instana, IBM Turbonomic, Palo Alto Networks, ServiceNow and Zabbix, according to a Red Hat press release.
"Rather than having another platform run our Ansible automation, we can create rules to proactively detect and respond to different problems in different roles within Ansible Automation Platform," Tellechea said in an interview this week.
It's tough going sometimes. You have to convince your senior executives of the necessity of investing your money, time and your team in automation and observability.
Marta CecilianoHead of middleware, Mutua Madrileña
Event-Driven Ansible's new policy-as-code rulebooks also offer finer-grained control than homegrown webhooks over when and how various workflows run in response to event alerts. Pre-built tie-ins will make it easier for more teams to follow the middleware team's lead without having to create the integration themselves, Tellechea said.
However, evangelizing the event-driven architecture within the company has been an ongoing, painstaking process since the middleware team began using Ansible in 2016, Ceciliano said during the presentation.
Within the middleware team, application owners had to be convinced to make automation part of the initial development of a project rather than something they would add after they created a manually managed version, she said.
Convincing each department's leadership to invest in event-driven automation throughout the organization has been a challenge as well, Ceciliano said.
"It's tough going sometimes. You have to convince your senior executives of the necessity of investing your money, time and your team in automation and observability," she said.
At health insurer Blue Shield of California, a similar movement is afoot to create an organization-wide automation process, which will likely include Event-Driven Ansible, according to SRE manager Ty Lim in an interview here this week.
"There have been some struggles but a lot of enlightenment" as Lim and his colleagues experimented with creating Ansible rulebooks. This required designing automation for a complete workflow rather than individual tasks through Ansible playbooks, he said.
"We're also moving to Azure [Red Hat OpenShift], and the sequence of how you deploy resources in Azure is important," Lim said. "We had some hiccups in the beginning with ... getting all the pieces to run so that the final product had everything that was requested."
Yamandú Tellechea and Marta Ceciliano of Mutua Madrileña present on Event-Driven Ansible at Red Hat Summit
Event-driven Ansible's AI roadmap
Integrating with Event-Driven Ansible rulebooks is a logical next step for the IBM Watson Code Assistant baked into the Ansible Lightspeed generative AI tool rolled out in public beta here this week, according to Red Hat officials.
"You want to be able to say [to the AI] model, 'Only let these things be run, but you can pick from them based on what data you see coming in,'" said Richard Henshall, director of product management in the Red Hat Ansible business unit, in an interview.
It will take time for most enterprises to trust generative AI, bu Tellechea said he's eager to get started with AI in Event-Driven Ansible.
"We already use the artificial intelligence provided by Dynatrace to detect anomalies in specific applications," Tellechea said. "In the future, maybe integration with artificial intelligence could create rules and interpret how to respond to events."
Red Hat partner and Accenture infrastructure manager Bryan Locke also said in an online interview this week that Event-Driven Ansible should prompt pricing changes -- a possibility a Red Hat spokesperson said the vendor is considering.
"I hope to see Red Hat … move to true consumption based -- only pay for what you use automation [and] avoid large year-on-year upfront commitments," Locke said, noting that he uses Ansible internally at Accenture and supports it for clients. "Treat automation like AWS treats Lambda: charge me when automation is triggered."
Beth Pariseau, senior news writer at TechTarget, is an award-winning veteran of IT journalism. She can be reached at [email protected] or on Twitter @PariseauTT.
