Large language models capable of autonomously detecting and exploiting software vulnerabilities have put cybersecurity professionals on high alert. For CISOs, the question isn't how to keep pace with emerging threats, but how to determine whether their security program is mature enough to defend against AI-enabled attacks -- and, if not, how to get there.
As demonstrated by models such as Mythos, LLMs can attack every component of an enterprise's applications and infrastructure. Not only can these models quickly detect and exploit known yet unpatched vulnerabilities, but they can also uncover and exploit previously unknown vulnerabilities, even in platforms that have been considered safe for decades. Examples include a 27-year-old integer overflow vulnerability in the OpenBSD OS and a 16-year-old flaw in FFmpeg, an open source media framework that handles audio and video encoding and decoding for most browsers, video players, streaming services, content delivery networks and other enterprise applications. Both OpenBSD and FFmpeg have been tested and retested millions of times, yet the vulnerabilities weren't found until now.
And these are just examples; Mythos has identified thousands of vulnerabilities that enable remote code execution and privilege escalation across every major OS -- Windows, macOS, Linux, BSD, iOS and Android -- and every major browser.
LLMs can also string together low-level vulnerabilities to produce high-impact attacks. This means that the standard cybersecurity operational approach of fixing vulnerabilities in order of priority will fail. Not only are there new, previously undiscovered high-priority vulnerabilities flooding the patch pipeline, but even low-priority vulnerabilities can't be ignored.
The bottom line: Nothing is safe, and existing processes no longer provide adequate security.
So, what should an enterprise cybersecurity team do?
First, determine how mature the organization is in protecting against AI-driven cybersecurity attacks. That will provide high-level guidance on risk level and drive decisions about what to do next.
Introducing the Nemertes AI Cybersecurity Maturity Model
At Nemertes, we use a standard maturity model to quickly and accurately categorize an enterprise's maturity on any given axis. The model consists of the following stages:
0. Unprepared. Cybersecurity teams lack the necessary information to take effective action and are unable to respond to current issues, let alone predict and protect against future issues.
1. Reactive. Cybersecurity teams have the platforms, structures and organizational processes to react to current technical issues and respond to current business requirements, but they can't proactively address these requirements or issues.
2. Proactive. Cybersecurity teams have the platforms, structures and organizational processes that enable them to address current business requirements and technical issues proactively, but can't predict -- much less prepare for -- future business requirements and technical issues.
3. Anticipatory. Cybersecurity teams have the platforms, structures and organizational processes to predict and proactively address future requirements and issues.
Achieving each stage beyond the first requires implementing critical technology, integrating it into the operating environment, and developing and perfecting the processes required to use the technology effectively.
Where to start: Getting to Level 1
Cybersecurity professionals should start assessing their maturity by completing the following three tasks.
Build a vulnerability heat map
A heat map shows where vulnerabilities exist. How much legacy code -- i.e., code more than five years old -- exists in the environment? In which systems? How much of the code handles sensitive data, is exposed to untrusted input or is connected to networks the rest of the enterprise depends on?
Note that this is required not just of homegrown or open source software, but for any system that runs software anywhere in the environment: routers, switches, storage devices and server OSes. That means asking vendors some pointed questions, as well.
If the answers to these questions are unknown and no heat map exists, the organization is at Level 0: Unprepared.
Develop a remediation plan
Even if the cybersecurity team knows the answers and has a detailed, presumably accurate understanding of the risk, the organization might still be at Level 0: Unprepared if there's no remediation plan or if the plan is unworkable at the scale at which attack LLMs operate.
For instance, manual replacement and code rewriting, which would require implementing redesigned systems by 2028 or 2030, will not protect an organization.
Revamp incident response processes
A pre-AI-era incident response plan cannot sufficiently protect an environment in the Mythos era. Processes must be accelerated, and priorities might be upended. For example, a process that remediates the top 50% of vulnerabilities within a week of discovery must be revamped to remediate all vulnerabilities within the hour.
That level of unprecedented speed requires automation, appropriate human-in-the-loop mechanisms and highly granular insight into the current state of the environment.
If the organization has an accurate heat map, a strong remediation plan and a solid incident response plan that's designed for the AI era, it's at Level 1: Reactive.
Moving from Level 1: Reactive to Level 2: Proactive
The next step is to move up to Level 2 in the maturity model. This involves proactively uncovering and addressing vulnerabilities, but doing so safely.
Many organizations have deployed Mythos internally as part of Project Glasswing to fire-harden their environments. This is wise. Running LLMs against your own production environment can be risky, but it's critical. The Machiavellian adage, "Keep your friends close, and your enemies closer," truly applies: Mythos is the enemy, so let it fire-harden your environment in a controlled fashion.
Implement effective, out-of-band configuration logging and auditing
Before moving to the next step, cybersecurity professionals must have an out-of-band system to track and manage configuration changes. These systems must let enterprises know who changed every configuration or line of code, and when. And more importantly, they must run out of band so they can't be modified by LLMs, which are known to make changes to hide their tracks. This will enable a successful rollback if the LLMs in the next step go rogue.
Run continual, proactive, automated testing using LLMs
Gone are the days of the annual or even quarterly penetration test. Cybersecurity professionals should monitor their environment on an ongoing basis using carefully controlled and managed LLMs. This is the only way to ensure that the vulnerabilities uncovered by the in-house test aren't also exploited by the model used to discover them -- given the well-known tendency of LLMs to exceed the limits of their instructions.
Remediate vulnerabilities in real times
As the proactive testing uncovers issues, the incident response plan developed earlier should go into effect, fixing things as quickly as they're discovered. These remediation efforts should include reconfiguring anything configurable and rewriting or modernizing any software that can be modernized.
Protect the unfixable
Not every device or system can be remediated. Protected these devices within layers that limit blast radius when the inevitable LLM-discovered CVE arrives. This includes deploying network segmentation, improving identity hygiene, adding application-layer firewalls and ensuring that logging occurs in real time.
Moving to Level 3: Anticipatory
Once the baseline systems and processes are in place, teams can protect not only against predictable vulnerabilities and attacks, but also against those that are likely to arise from the future evolution of LLMs and other technologies -- including quantum. That's what's necessary to move from Level 2: Proactive to Level 3: Anticipatory.
This is a two-pronged approach:
- Prong 1 comprises a series of hardware-based kill switches that can quickly and effectively shut down any LLM-based attacks, and which operate out of band -- not reachable in any way by any enterprise software. This technology doesn't exist yet, but it's on the horizon -- look at Nvidia's Open Agent Safety Platform.
- Prong 2 comprises the ability to determine on a case-by-case basis, in real time, when an action being taken by an LLM might result in harm. This technology doesn't currently exist, but it's coming, likely based on quantum computing.
Zero trust is the answer (for now)
Given that neither prong currently exists in an implementable form, the workaround is to implement zero-trust security at the most granular level possible. Zero-trust is the marketing term for an approach to security that essentially involves allowlisting all permissible interactions.
In other words, instead of having all forms of interaction permissible by default and turned off by policy, it's the other way around: No system can interact with any other by default, but only if the interaction between those two systems is explicitly authorized at this specific point in time or current state.
This involves putting three elements in place:
- A strong identity platform. It encompasses both humans and LLM agents and is designed to handle ephemeral identities -- e.g., an agent that's created and terminated in a matter of minutes.
- A solid allowlisting policy. This requires thinking through every possible interaction and determining whether it's permitted by a particular entity at that point in time. For example, an entity might have admin rights to make configuration changes to a particular piece of software, but only while there's an active trouble ticket to which that entity is assigned. If the ticket is closed, the entity should lose access.
- Comprehensive, out-of-band controls. This requires implementing -- in software, if necessary -- an out-of-band system that shuts down potentially dangerous interactions when they're discovered.
Conclusion
The journey toward solid AI cybersecurity maturity starts with awareness: understanding the true nature of the changes that have occurred in recent months and their impact on systems and processes.
Implementing those systems and processes in a structured fashion -- starting with a heat map and ending with zero trust -- is essential to ensuring AI cybersecurity maturity. In other words, security maturity is about building appropriate, measurable and risk-aligned controls, not simply accumulating more controls.
Johna Till Johnson is CEO and founder of strategic consulting and research company Nemertes, where she sets research direction and works with strategic clients.
