What CISOs should take from the Hugging Face-OpenAI incident
Security experts say the lesson isn't to abandon sandboxing, but to strengthen the security controls around surrounding systems as AI tests their limits.
The recent Hugging Face-OpenAI incident is raising questions about how to secure AI as it becomes more autonomous and integrated into business operations.
During a controlled security exercise in mid-July, OpenAI models accessed systems they weren't supposed to reach by exploiting a vulnerability in the surrounding infrastructure, eventually reaching the Hugging Face AI development platform.
The incident has challenged assumptions that isolation and sandboxing can sufficiently protect AI systems. Yet security experts say the bigger takeaway is about whether businesses have properly implemented fundamental security practices such as identity and access controls, monitoring and containment.
The sandbox worked -- the environment didn't
"The sandbox worked exactly as designed," Jen Waltz, founder and CISO at Imajenative, a Chicago-based IT and cybersecurity consultancy, told TechTarget Cybersecurity. "Unfortunately, the environment around it did not. That distinction is the whole lesson."
In the Hugging Face-OpenAI incident, Waltz added, AI didn't reinvent the wheel; instead, it showed how quickly an AI system can exploit existing security weaknesses if it has a goal and access to pursue it.
"AI didn't invent a new class of attack," she said. Rather, it executed the old ones at speed, before human operators noticed or stopped it.
"I do not agree that this challenged traditional sandboxing assumptions," echoed Rich Mogull, chief analyst for the Cloud Security Alliance (CSA), when we asked him about the limitations of this approach to security. "What we saw was a sandbox with a hole, and a system that appears to have been unmonitored."
The takeaway for CISOs: Don't abandon traditional security controls; instead, ensure you're applying them effectively as AI systems find new ways to gain access and take more actions on their own.
Steps CISOs should take now
CSA this week issued a post-mortem report on the Hugging Face-OpenAI incident, recommending three steps for CISOs to take to prepare for AI-driven incidents.
What we saw was a sandbox with a hole, and a system that appears to have been unmonitored.
Rich MogullChief analyst, Cloud Security Alliance
Now: Identify and secure AI agents that are at the highest risk. Limit unnecessary permissions and confirm teams can shut down risky activity.
This month: Monitor AI behavior and make sure systems can recover quickly when something goes wrong. Deploy and test deception technologies and AI incident response processes.
This quarter: Prepare for AI incidents before they happen by assigning responsibility for AI systems to someone who will respond when they behave unexpectedly. Run AI tabletop exercises and deploy system-wide deception technologies.
A major challenge for CISOs is how quickly AI systems are becoming connected to more tools and information. Security teams need to know what they can access and how to respond when AI doesn't behave as expected.
SANS Institute recommends that security leaders reconsider how they manage AI systems as those systems gain access to sensitive data.
"An agent is not a user, and it is not a service account," said Rob T. Lee, chief AI officer and chief of research at SANS. "It is closer to a brilliant intern with infinite energy, no instinct for boundaries and whatever credentials you handed it."
While AI providers continue to improve built-in safety measures, Lee cautions against confusing those controls with security enforcement. "Guardrails are etiquette, and access control is law," he said.
For security leaders, that means moving beyond merely asking if AI can be secured and focusing on how their organizations can understand what AI is doing and who is accountable for its actions.
"The question was never whether organizations should adopt AI," Waltz said. "That decision was made without most security teams in the room. The advantage will belong to the organizations that can prove what their AI did, why it did it and who owns the outcome."
Craig Galbraith is the founder and owner of Galbraith Multimedia, an independent journalism company that provides writing, editing, video hosting, podcasting, onstage presentation and consulting services to the technology industry.
