Attackers are adapting social engineering tactics to AI agents, planting instructions in documents and software repositories that persuade the systems to install unwanted software or expose sensitive information.
Recent examples show how malicious instructions disguised as routine work can turn an agent’s own access against it, from a poisoned file sitting in a customer’s downloads folder to a compromised credential pipeline inside an open source project.
An agent carrying out routine work can encounter those instructions with legitimate access to enterprise systems already available. The manipulation exploits a failure to distinguish material the agent should analyze from directions it should obey.
Bobby Kuzma, senior research manager for DevSecOps at market intelligence firm IDC, said viewing these attacks as a form of social engineering can help security teams identify controls that can contain the consequences.
“People — and now models — fail in the same ways, and it boils down to boring and unsexy controls like separation of duties and segmentation to limit the blast radius,” he said.
Those safeguards matter even more because an agent’s next task won’t retain lessons from an earlier deception attempt. Security teams cannot assume that encountering an attack will improve the model’s response to the next one.
“People learn from near misses, but models only have what you give them, so they’re potentially starting tabula rasa with every new request,” Kuzma said.
What separates a contained incident from a supply chain breach often comes down to decisions made long before an agent ever reads a malicious instruction, and how quickly responders can trace what it touched after the fact. With this in mind, security teams are reconsidering how authorization and containment controls need to work when the thing being deceived is a model rather than a person.
Malicious instructions arrive as ordinary work
Prompt injection attacks exploit an agent’s handling of instructions embedded in material it should treat as data. Describing the attack as a form of social engineering (or, perhaps, “system engineering”) also captures how the material acquires apparent credibility.
Diana Kelley, CISO at Noma Security, said attackers can impersonate trusted sources, manufacture urgency or present a malicious step as necessary to complete legitimate work.
“With agents, it’s about getting lower-trust content treated as authority,” she said.
Trusted instructions and untrusted documents or tool responses can enter the same model context, Kelley explained. Screening can reduce malicious material reaching the model, but independent controls must limit the consequences when screening fails.
In October, Forever Security reported finding a malicious AGENTS.md instruction file in a customer’s Downloads folder. A coding agent followed it and attempted to install a malicious software package. The company said its monitoring stopped execution.
The researchers did not establish how the file arrived. Subsequent testing reproduced attacks against coding agents, but those demonstrations were separate from the blocked customer incident.
The file disguised its instructions as routine engineering guidance, illustrating how an attacker can make an unwanted action appear to be preparation for an authorized task.
Maxime Cartier, vice president of human risk at Hoxhunt, said defenders should consider the campaign surrounding an instruction, including reconnaissance and efforts to influence whoever approves the agent’s work.
“Prompt injection describes that one malicious instruction. Social engineering describes the campaign around it,” he said.
Trace the attack through to its outcome
A similar pattern played out beyond coding assistants. Cline, an open source AI coding assistant, disclosed in February that malicious instructions submitted to its automated issue-review process could expose software-publishing credentials through a shared cache.
An unauthorized party later used a compromised token to publish an altered package. Cline reported no malicious code delivery or user-data theft in its post-mortem.
Pyry Åvist, co-founder and CTO at Hoxhunt, said incident assessments must track whether malicious content reached an agent, whether it followed the instructions and whether a sensitive action succeeded.
“Calling every successful laboratory prompt a breach obscures where the real risk currently stands,” he said.
That approach also helps responders identify which defenses held. An agent attempting an unauthorized installation and an endpoint control blocking it are different outcomes within the same attack.
Enforce authorization outside the model
Security controls must remain effective even if an agent accepts the attacker’s instructions, Kuzma said.
“If you operate with the assumption that a model will do exactly what an attacker says, then you should build so that doesn't matter,” he said.
Agents need separate identities and short-lived credentials limited to their assigned tasks. Keeping standing secrets away from their execution environments reduces what an attacker can obtain, while outbound restrictions can block unauthorized transfers.
Kelley emphasized separating the model’s requested action from the system’s decision to permit it.
“The model may follow a malicious instruction,” she said. “That doesn't mean the enterprise has to execute the resulting action.”
Åvist called for separating read and write access, restricting execution and independently approving consequential actions such as publishing code or changing permissions. An agent reading a document should not automatically gain authority to install software or retrieve unrelated secrets.
Kevin Surace, CEO of TokenCore, argued for hardware-backed biometric approval of critical actions, tied to the specific transaction and enforced outside the model. That is his proposed implementation of human authorization, rather than a requirement shared by all the experts.
“The approval must also be tied to the specific transaction, so the human knows exactly what is being authorized,” he said.
Contain delegated work and persistent changes
An agent with access to source code and cloud infrastructure could turn a poisoned document into a software supply chain incident, Kuzma cautioned. Delegation adds another route for malicious instructions to spread when one agent’s output becomes another’s input.
Responders therefore need to trace downstream jobs, revoke delegated credentials and identify any additional access created during the incident. Stopping the initiating agent alone may leave other tasks running.
Kelley said containment must also examine shared memory and persistent state for malicious instructions. Åvist identified generated code, documents and retrieval indexes as material that might require investigation before workflows resume.
Logs need to connect the original request and retrieved content to tool calls, credentials, resulting changes and delegation. Without that record, responders cannot establish which systems were reached or whether unauthorized work continues.
“If your logs can’t show who delegated what, you can't scope the incident, and you'll be guessing,” Kuzma said.
