putilov_denis - stock.adobe.com
What CISOs should know about AI runtime security
With AI, security at runtime means protecting access, vetting inputs, checking outputs and detecting anomalous behaviors. It is challenging to get right.
CISOs recognize the cybersecurity implications of AI, but many remain focused on preventing AI-enabled data loss and compliance breaches. Few are paying attention to the full scope of AI-related cybersecurity yet.
Runtime security focuses on protecting running models from compromise. This means monitoring, protecting and controlling AI systems while they are actively running and making decisions -- rather than only securing them during development and deployment.
Preventing compromise of a running AI tool protects the organization not only from data leaks and compliance breaches, but also from AI being used as a weapon to launch or assist in other attacks. It protects the AI tool and it protects the business from the AI tool. It safeguards models against threats such as prompt injection attacks, unauthorized tool use, excessive permissions and model abuse.
Protection at runtime requires safeguarding access, vetting inputs and checking outputs. It also means watching AI for anomalous behaviors.
Beyond the oft-cited data-leak scenarios, CISOs might wonder why in-depth runtime protection is worth the effort. To get into the right mindset, think of it not just as software, but as staff. A traditional web application generally does only what developers code it to do. AI models and agents, on the other hand, can reason, choose tools, chain actions together and perform actions with multiple outcomes, depending on how they are trained. A subverted AI can damage the enterprise in proportion to how much it is trusted and by whom. Consider these scenarios:
- An analytic AI in the network that helps staff troubleshoot problems can also provide key compromise intelligence to an external bad actor. It can also be used to help hide indicators of compromise.
- Agentic AI in the network that is capable of changing network device configurations can create security holes for external actors. It can also divert or stop streams of monitoring data that would indicate ongoing compromises.
- An AI application with no vetting for prompts or outputs can be fed prompts or external data that contains embedded instructions directing it to exfiltrate data.
- An AI application with no prompt vetting can be tricked into lying in response to legitimate prompts and employee requests. Workers are not used to thinking that their data tools could intentionally misrepresent available data.
Challenges of AI runtime security
The biggest challenges CISOs face in securing AI at runtime are the expanding uses of AI within the enterprise, how the technology is evolving and a lack of AI-specific tooling.
AI use is evolving fast
Software vendors are still finding different ways to weave AI into their wares. Users and organizations, meanwhile, seek value by wrapping AI around or interposing it between systems. Organizations are giving AI tools access to more data, more kinds of data and more of the core IT service environment. Inserting security in all the necessary places, therefore, means trying to hit a moving target.
AI's core technologies are evolving
The most obvious -- and from a cybersecurity perspective, the most dangerous -- changes come in the rapid shift from passive to active tools -- agents -- and from isolated AI tools to integrated ones. This latter shift, largely facilitated by the rapid rise of the Model Context Protocol (MCP), complicates and expands threat surfaces.
Lack of AI-specific security tooling
The absence of AI-specific security tooling puts the burden on older security tools and services, which are not up to the challenge. Conventional static code scanners and software composition analyzers won't detect corrupt prompt files or skills definitions, and traditional web application firewalls can't detect or block malicious prompting of a web-facing AI.
It's crucial to keep in mind, too, that attackers also have access to AI and use it to find ways to compromise AI tooling.
Best practices for securing AI at runtime
Above all, AI needs zero-trust security. From there, many other best practices become apparent, including the following:
- Put identity at the core of zero trust for AI. In practice, this means applying identity not just to users and conventional software and hardware systems, but also specifically to AI.
- Apply zero-trust principles to AI. Block all access to any AI tool that has not been specifically authorized. Gate access to users and systems, including other AIs via MCP, that the tool is meant to serve.
- Extend zero trust beyond raw access. An entity allowed to reach an AI system shouldn't by default be able to see or use all parts of its functionality -- unless that is the intent. For AI, protection at this level includes prompt filtering to block inappropriate data requests or attempts to subvert the tool.
- Apply zero trust to runtime behaviors. Any full zero-trust environment requires the organization to monitor behavior and actively update access privileges based on it. With AI, this might mean blocking users who repeatedly try to feed suspicious prompts, and possibly also the systems they connect from. Likewise, block any MCP nodes that try to get the AI to misbehave. Shut down access to and from AI tools that themselves begin to act strangely or maliciously.
Enacting these principles in the environment requires implementing several new layers of cybersecurity tooling, either by acquisition or upgrade. It could also require an identity management system capable of handling a highly dynamic AI agent environment. CISOs should conduct a risk assessment based on the organization's AI strategy to prioritize which of those necessary investments to make first.
John Burke is CTO and a research analyst at Nemertes Research. Burke joined Nemertes in 2005 with nearly two decades of technology experience. He has worked at all levels of IT, including as an end-user support specialist, programmer, system administrator, database specialist, network administrator, network architect and systems architect.
