LLM firewalls emerge as a new AI security layer

What's an LLM firewall?

With the coupling of AI and operational systems come the risks of prompt injection attacks, model poisoning, data leaks and dangerous misconfigurations.

LLM firewalls have emerged as one way to counter these risks. The tools enable security teams to monitor, filter and sanitize user input, manage how a model interacts with other systems and understand how data might flow through it.

One of the specialized firewall's primary functions is to protect the LLM against prompt injection attacks -- where an adversary crafts inputs that manipulate the model into performing unintended actions or responding outside its safety guardrails. Firewalls for LLMs also aim to protect against other risks, including data leaks -- for instance, by preventing users from inputting sensitive data into the model; malicious code generation; privilege escalation attacks; and model overuse.

How LLM firewalls are different

LLM firewalls differ from web application firewalls (WAFs), which inspect message content for indications of code injection and other types of attacks. They also differ from lower-level network firewalls, which make security decisions based on port numbers, protocols and other patterns in network traffic.

"Each has its place in a security architecture, but an LLM firewall is increasingly necessary as organizations roll out their own LLMs and LLM-enabled applications that require specialized protection that WAF and network firewalls cannot provide," said Christopher Rodriguez, research director of security and trust at analyst firm IDC.

Rik Turner, an analyst at Omdia, a division of Informa TechTarget, said to think of AI firewalls as tools that analyze the semantics, intent and context of natural language as contained in both incoming prompts and outgoing responses.

Such firewalls typically have three distinct components or layers, Turner said: a prompt firewall that scans user input before it reaches the LLM to block jailbreaks, prompt injections and malicious commands; a retrieval firewall for managing data fetched from external databases during retrieval-augmented generation; and a response firewall for outbound traffic, which reviews the model's generated text before it reaches the user.

The LLM firewall market: A feeding frenzy?

Several established vendors, including Palo Alto Networks, Cloudflare, Akamai, Varonis and Check Point, have begun offering LLM protection capabilities as part of their broader security portfolios. There's also a rapidly growing list of vendors that offer specialized LLM security products, including Lakera, Prompt Security, HiddenLayer and CalypsoAI.

Richard Stiennon, chief research analyst at cybersecurity market intelligence firm IT-Harvest, pointed to several other vendors in the broader AI security space that also offer firewall capabilities for LLMs. Examples include Operant AI, Aiceberg, Acuvity, HydroX AI, Cytex and Citadel AI.

Estimates of the current size of the LLM firewall market vary widely, reflecting the early and still-emerging nature of the category. IT-Harvest has pegged the current market for AI firewalls at a modest $30 million and estimates the segment will grow 100% in 2026. Others have higher projections. 360iResearch, for example, estimated the market size at $260 million in 2025 and slated it to hit almost $800 million in 2032.

A nascent technology: Too soon to say

The segment is so new that not all vendors are even settled on the term LLM firewall, Stiennon said. Stiennon himself listed them under what he calls the "model protection" category. Others, he said, might refer to them as AI firewalls.

From an effectiveness standpoint, Turner said many of the currently available AI firewalls offer reasonably good protection against jailbreaks, prompt injections and malicious commands. They can filter content that users might input into a model to protect sensitive data and personally identifiable information. They also do rate limiting to throttle DDoS attacks against the model and the server on which it is hosted, Turner said.

But they may struggle to detect newer forms of attacks, he cautioned. "A lot of the current generation of LLM firewalls analyze prompts individually, which means they lack context across multiple prompts," he said. They could therefore struggle to detect stateful or conversational attacks, in which an attacker might gradually manipulate a model over several interactions to bypass security rather than using a single malicious prompt.

It's also still too early to draw definitive conclusions about the long-term effectiveness of LLM firewalls, given how new the technology is and how recently organizations have begun deploying it. Attacks targeting AI environments are also constantly evolving, so there's no telling what additional security controls will be needed to address them.

"LLM firewalls, aka firewalls for AI, inspect the interactions -- both inbound and outbound -- with an LLM or LLM-enabled application," IDC's Rodriguez said. "These checks often require the ability to understand meaning, context and intent of messages."

This ability will be key to effectiveness, said Michael Smith, field CTO at DigiCert. Without context, an LLM might be poisoned with misinformation, and there is no way for the LLM firewall to identify this.

"Or the LLM could hallucinate, or recite inaccurate facts, which are not dangerous to the LLM, the data inside of it or the user's client. But it is dangerous to the human who takes the hallucination as fact and acts based on that," Smith added.

Do organizations need specialized firewalls for AI?

Organizations need to know exactly what they want to protect against and where to deploy these controls. Decision-makers should answer the following basic questions to derive real value from their AI firewall investment, Smith said:

• Where is the LLM hosted, and does the firewall deployment model support that?
• What kinds of data does the firewall have to be able to recognize in a prompt or an output?
• Where and how will the output of the LLM be used?
• Do you need to protect the LLM client or things that it controls?

With so many AI firewall options readily available -- many from startups and companies with little to no track record in enterprise environments -- making purchasing decisions can be hard. So, knowing what to look for and what to ask can be crucial. Rodriguez stressed the importance of decision-makers paying attention to two factors in particular: accuracy and latency.

An AI firewall with too many false positives can frustrate users, while one that is prone to too many false negatives can expose the organization to heightened business risk, he pointed out.

"Accuracy of detections will become ever more important as organizations begin to better understand the business risk surrounding their LLMs and LLM-enabled applications," Rodriquez said. Latency is also important because many LLM firewall offerings are cloud-based, he added.

At the end of the day, while LLM firewalls are likely going to be an important requirement for organizations harnessing GenAI technologies in their operations, they are only part of a broader stack of needed security controls. True defense-in-depth for AI security means deploying capabilities for broader AI security posture management, data loss prevention and data security posture management for both training and inference data, Omdia's Turner said. Also likely needed are tools for tokenizing sensitive data so no private data is exposed in an AI model, he noted.

"Generative AI right now is the killer shadow IT application," DigiCert's Smith said. "It has trickled into so many applications and workflows now that it's impossible to keep it out of your organization."

Jaikumar Vijayan is a freelance technology journalist with more than 20 years of award-winning experience in IT trade journalism, specializing in information security, data privacy and cybersecurity topics.