Alex - stock.adobe.com
DeepSeek and what enterprises should learn from AI agent exploits
A new case of an out-of-control AI model shows that enterprises should treat models and agents as toddlers that need supervision and explicit instructions.
In another instance of an AI model taking unsanctioned action, Chinese vendor DeepSeek’s model v4.1 Flash used API keys sitting in its own sandbox and used them to call out to other models.
The incident is a reminder to enterprises not to leave AI models or agents without effective guardrails or guidance.
Two separate coding agent benchmark tests showed the v4.1 Flash model taking exposed keys left in its environment, rather than ignoring or reading them. The model used the keys to call out to other models and inflate its performance scores, according to reports earlier this week.
The DeepSeek incident is one of the first reported cases in which a Chinese model has acted without permission in a testing environment.
On Oct. 7, CrowdStrike revealed that a Chinese open source agentic testing tool called ARTEX and large language models from China targeted South Korean financial organizations from late September to early this month.
Those incursions followed revelations by U.S. generative AI vendors that their models made decisions on their own, without prompting. OpenAI revealed over the summer that its models escaped a sandboxed testing environment. In another instance, an OpenAI agent used stolen credentials to compromise Hugging Face’s infrastructure. Anthropic and Google have also admitted that their models left testing environments on their own.
“These are very powerful tools, and we have to be careful with them,” said David Nicholson, an analyst at Futurum Group. “I would liken these models and the way that we need to treat them to the way that we treat an environment where there are children of varying ages and dangerous objects.”
Models as children
Models and agents need close guidance and clear instructions to prevent them from acting on their own, regardless of whether the model maker is from China or the U.S., Nicholson said.
“We know that these models, if you leave something within the context window that they have access to, they’re probably going to use it unless you somehow construct an environment that explicitly prohibits them from doing it,” he said.
So, the misbehaving models from DeepSeek, OpenAI and Anthropic should signal to enterprises that they need to “childproof” the environments in which the models are deployed, particularly context windows.
“We need to be more responsible with these things that we are playing with, and we shouldn’t let them have access to things in as careless a way that we are,” Nicholson added. “We should expect the models to behave in a way that creates unintended consequences.”
He added that enterprises should strengthen security and provide agents with unambiguous, detailed instructions on what they can and can’t do. Enterprises should also assume by now that models and agents can’t be trusted unquestioningly.
“Don’t leave anything in a context window or accessible to one of these models if you’re not okay with it using it for whatever it wants to because it will,” Nicholson added.
Esther Shittu is an Informa TechTarget news writer and podcast producer and host covering AI technology.
