As federal government agencies deploy AI for a wide range of tasks, they’re discovering that managing inference costs, protecting sensitive information and ensuring the technology serves the mission all present particular challenges. For many complex workloads, the latest proprietary frontier models may be worth the expense. For others, an open-weight model running in a government-controlled environment can deliver sufficient performance at a lower operating cost.
Public sector and cybersecurity leaders at October’s 2026 Red Hat Government Symposium in Washington, D.C. discussed the benefits of using a hybrid approach that combines frontier models for complex analysis and mission-critical reasoning with smaller, less expensive models for routine tasks.
The goal, according to panelists, is to spend AI resources where they can have the most impact, while maintaining control over data, security and topics of national interest.
Red Hat’s chief architect for health and life sciences Ben Cushing said a hybrid approach lets agencies control costs without sacrificing access to top-tier AI capabilities.
“Adopting a hybrid cloud strategy and running these open-weight models locally on infrastructure you control is honestly the safest way to go about creating an environment where you can take advantage of all this exciting system development and, at the same time, get some of the latest and greatest frontier lab capabilities,” he said.
Why open-weight models are gaining ground
Open-weight AI models have trained parameters — or “weights” — that users can download, deploy and, depending on the licenses, modify or fine-tune. Unlike proprietary models, which typically require access through a vendor’s hosted platform or API, open-weight models can be deployed on an agency’s own infrastructure or through an approved cloud provider. Examples include models from Meta’s Llama family, Alibaba’s Qwen and DeepSeek.
In government agencies, where budgets can be tight, use of AI models often becomes a financial consideration. For organizations deploying AI across thousands of employees or processing large volumes of documents, charges can accumulate quickly. Long-context prompts, repeated requests, large document collections and automated AI workflows can all increase consumption.
“Just last year, speaking with CIOs across the federal health space, they were describing to me how they went through their cloud credits on tokens within a matter of a month,” recalled Cushing. “They planned out the whole year. They got their budget and a number of developers burned through the tokens so fast that they didn’t have any way to balance their budget.”
Using open-weight models, agencies still consume tokens, and inference still requires computing resources, but they are not necessarily paying a vendor a fee for every token processed. Instead, they pay for the infrastructure, hosting, maintenance and personnel required to operate the model.
The case for a hybrid model strategy
Not every government AI application requires advanced reasoning capabilities. Summarizing routine documents, extracting information from records, classifying correspondence, searching internal knowledge bases and identifying patterns in structured data are examples of tasks that may be handled by smaller or less expensive models.
Cushing suggested that the ideal scenario would be one in which the AI platform would select the model based on the use case. For something straightforward like transcribing interviews in a criminal investigation, open-weight models would suffice.
“But there may be other very specific fraud analyses that data scientists are looking to do that would be worth investing some tokens or some money to make sure I can do that,” he said. “If I know I’m sending that particular use case [to the right model], then I can plan against that.”
The Department of Homeland Security’s January 2025 Playbook for Public Sector Generative AI Deployment explicitly recommends that agencies evaluate commercial, open-source and open-weight models, including the possibility of using multiple model types.
At the same time, the National Institute of Standards and Technology has sought access to open-weight model hosting and inference services for its Center for AI Standards and Innovation. Its requirements reflect the government’s interest in evaluating open-weight models as part of its broader AI and national security work.
Lance Jenkinson, CIO and deputy assistant inspector general for management administration at the Department of Veterans Affairs, shared that his agency has created 42,000 patches over the last several months, primarily using open-weight models.
“We’ve got about 300 AI-driven processes, and 85% of those today are using open-weight models versus AI frontier models,” he said. “You can get answers that are just as accurate, more often than not, with those open-weight models at a far, far lower cost.”
Proprietary AI isn’t going away
Open-weight models aren’t the only way government organizations control spending. Purpose-built proprietary AI services, government-specific pricing agreements and consumption-based contracts can also reduce costs.
The General Services Administration has pursued discounted AI agreements through its OneGov initiative, giving government buyers access to commercial AI services under negotiated terms. In October 2026, GSA announced an expanded agreement with Anthropic that provides a 40% discount on token-based usage across Claude models available to government customers.
Agencies therefore have more options than simply choosing between expensive commercial models and free downloadable weights. They can negotiate commercial pricing, use models already available through approved cloud contracts or operate open-weight models on infrastructure they control.
Jenkinson shared that each agency needs to weigh mission effectiveness against the capabilities of the AI, asking: “Is it secured? Is it supported? Then, perhaps, some open-source tools get [agencies] 90% of the way there at a fraction of the cost.”
No one model to rule them all
The federal government’s approach to AI is increasingly about flexibility. Agencies report needing access to advanced models, with the caveat that many tasks require smaller, less expensive alternatives.
Panelists recommended that decision-makers evaluate models based on the work they need to perform, recognizing that proprietary AI can be cost-effective under the right contract, just as open-weight AI can introduce infrastructure and maintenance expenses that offset some of its apparent savings.
“In the AI space, there is absolutely not one model that rules them all,” said Cushing. “Every model is better at certain things than others, and you’ve got [to consider] the cost elements and the accuracy elements.”
Richard Livingston is an editor for TechTarget Cybersecurity, covering news, trends and analysis. Livingston’s professional background includes editorial positions in the national defense industry covering the U.S. Army Medical Department, as well as offensive and defensive cyber strategy for military and government audiences.
