Over roughly 36 hours, a network of AI agents running on one engineer’s account spent about $500,000 setting up infrastructure in a cloud platform. Nobody broke in. The agents inherited the account’s broad-scoped privileges, moved faster than any person could and had no mechanisms in place to cap spend. It’s one of the bigger incidents I’ve seen this year, and the builder was someone who practiced secure development.
That was a technical builder, yet most agents today aren’t built by engineers. Employees build their own in tools like ChatGPT, Gemini or Claude and connect them directly to company systems, often without IT’s approval. A sales rep might build an assistant to triage email and update the CRM, connecting it with their own login. That assistant can now reach everything the rep can, and most employees in that position don’t realize what they’ve handed over — or that anyone who compromises the agent gets the same access.
In a Gravitee survey of more than 900 executives and technical practitioners, 88% of companies reported confirmed or suspected AI agent security incidents in the last year, and only 14% said all of their agents go live with full security and IT approval. Chasing agents down after launch leaves IT permanently behind. An agent running on an employee’s login looks like that employee, and it acts faster than anyone can shut it off. The limits on what agents can do, which identity they use and what they can connect to have to be set before employees start building.
Why early decisions determine most agent risk
Whether the builder is an engineer or an employee automating part of their own job, every agent starts with a few decisions that establish its risk profile. The first is whose credentials it acts with: its own identity, the login of the employee who built it or a service account IT creates specifically for software. The builder also decides what the agent can do, what it connects to and who can use it. Those decisions are typically made once and then forgotten. An employee who gives an agent access to their inbox rarely revisits that grant, even after the agent becomes part of a larger system where other agents use it in ways the builder never planned for. Builders can’t be the control point for risks they’ve stopped seeing.
Attackers are already exploiting this vulnerability. When an agent can read an employee’s inbox, anyone who emails that employee can send instructions to the agent, which makes prompt injection as simple as sending a crafted email. If the same agent can write to the CRM, one crafted email can change every record before anyone thinks to shut it off. I’m also seeing more credentials compromised through AI systems than in any prior year because most people building agents skip basics like storing keys in a credential manager. Each of the four steps below puts a guardrail around one of those early decisions before the first agent ships.
Step 1. Restrict what agents can do before anyone builds one
Agents shouldn’t start with default privileges for any action. Least privilege applies, and I think about permissions in stages: read access to safe things at one end, write access to production systems or data at the other, where the most care is required.
Most actions need a human in the loop until you have high confidence they’ll work fully automated. Moving money, deleting data or infrastructure, and sending anything outside the company all belong there. Spend shows where the line falls. A hard budget cap should be deterministic and automatic — it’s the control that would have stopped that $500,000 incident long before it got there. But when runaway spend signals a compromised account, shutting the account down is consequential enough that a human should approve it, unless the blast radius is trivial.
To decide where an action falls, treat it as a business underwriting question: Can the business accept the loss if the AI gets it wrong? Remove AI from the scenario and ask whether you would delegate that decision to someone downstream of you. If you wouldn’t hand it to a person, don’t hand it to an agent.
"Remove AI from the scenario and ask whether you would delegate that decision to someone downstream of you. If you wouldn’t hand it to a person, don’t hand it to an agent."
Vinay Thakker
Co-founder and CTO, KloudStax
Step 2. Give every agent its own identity
IDC research found that nonhuman identities outnumber people by as much as 75-to-1 in some environments. At that scale, an agent running on its builder’s login is effectively invisible because its activity looks identical to the builder’s. If two agents, a person and another automation all share one key, IT has no way to tell them apart, and unexpected behavior hides in the noise.
Shared keys also slow the response to a compromise. IT’s only option is to rotate the key, and every agent and system using it stops working until someone updates it. Giving each agent its own identity lets IT trace every action to a specific agent and shut one down without taking the others with it. You can’t contain what you can’t tell apart.
Step 3. Control which systems agents can connect to
Connections should follow business objectives. Before connecting an agent to a system, teams should define the problem it solves, how they’ll measure results and which data it actually needs. Without that discipline, teams keep bolting data, tools and access onto an existing agent until it can reach nearly every system in the company.
That doesn’t mean starving agents of context, which is what makes them useful. Be generous with read access to curated data sets prepared for agent use — ones that exclude customer information and anything else sensitive. Treat agents the way you’d treat people: scoped access to what the job requires, and open access to what’s safe.
Step 4. Give employees approved tools to build with
Companies run into trouble when they don’t give employees approved systems to connect to because employees then connect their agents to everything they can reach. Approved options might include Model Context Protocol tools the company builds, custom agents or databases prepared for agent use stripped of sensitive data and limited to approved actions. With those in place, IT doesn’t need to review every new agent, and employees can build quickly within limits the company sets.
Reviewing every agent doesn’t scale, and employees won’t wait for the review. Approved tools that are easier than the alternative do scale because the safe path becomes the fastest one.
Why this matters now
A company that adopts AI and does nothing will end up with thousands of agents accumulating over time and increasingly connected to each other. Assistants now find and call whatever tools are connected to them, so at some point yours will use an agent to do something you forgot it could do. Every agent with open access is another way in.
The hyperscalers already provide the building blocks for company-governed agents, and turning them on is configuration work, not a research project. Many companies simply haven’t started. For agents already in use, start by giving each one its own identity, which shows what every agent does and how often without disrupting the workflows people depend on.
Set the controls upfront, and you may end up with 100 well-built agents instead of 10,000 nobody can see.
Co-founder and CTO Vinay Thakker brings over a decade of experience in the Google Cloud ecosystem to KloudStax. Beginning his Google Cloud journey in 2009, when Google first entered the enterprise arena, Vinay has unmatched tenure leveraging GCP for business transformation. As a former CTO and founder of a top-performing Premier Google partner, he has a proven track record in driving innovation and delivering measurable ROI for clients operating on cloud. His deep understanding of cloud infrastructures and commitment to client success empower our customers to fully realize the benefits of cloud solutions.
