AI apocalypse warnings: Safety concern or market strategy?

Map the positions to the business models

How do we get there? We map the positions to the business models. Let's start with who disagreed. The U.S. president quickly stepped in, calling the Coxon/Amodei, Altman and Musk warnings a conspiracy. His concern is likely geopolitical rather than anything to do with the business models and any threats they might pose. Nvidia's Jensen Huang, who has a lot of skin in this game, urged the U.S. to move as quickly as possible, and Meta's Mark Zuckerberg made a similar case when he appeared at Dreamforce the following day.

This isn't at all difficult to unpack as we look at the business models on each side. The companies asking for restraint are selling metered access to frontier models. They make money by charging customers for the use of their most advanced models and succeed by either being -- or being perceived as -- a better solution than anyone else.

But this is a fast-moving space, and these companies are increasingly threatened by low-cost, attractive open-weight models that are quickly closing the gap. While it's possible their safety concerns are real, there's also a very compelling argument that a coordinated slowdown benefits the major players and removes the stigma or the impression that any are weaker than the others -- keeping the playing field level.

The companies poo-pooing restraint are the ones who make money differently. Nvidia sells the chips that the world runs on -- and that every AI lab trains on. Slowing that train down hurts revenue. Meta has a different AI strategy built around releasing most of its primary AI models for free rather than charging for access -- a move designed to commoditize the AI market, prevent rivals from locking down the ecosystem and keep enterprise data decentralized. This means Meta doesn't depend on frontier pricing. A "pacing agreement," as advocated by the closed labs, would mean constraining Meta without providing any protection for its revenue stream. I should note here that with Meta's newer AI agent tech Muse Spark, the company is selling commercial and API access rather than providing it for free.

What did we collectively miss? The antitrust bit

In the uproar following Coxson's warning posts and Amodei's article, there is an important part of the conversation that I believe we collectively missed. Amodei didn't ask for a slowdown for safety reasons -- he asked for a narrow antitrust waiver so rival labs could collaborate without running up against collusion laws. One bit of legal commentary I read on this front said it succinctly: Competitors agreeing on how fast to build is, legally speaking, a cartel. Super.

They all continue apace, burning capital, marching toward IPOs, and trying to get to profitability. If any of them slow down, they risk being labeled weak by the market, triggering investor anxiety. Perhaps the only way to slow this runaway train is to raise the alarm about safety, get a government-sanctioned "pace pact" in place and then spend less so it doesn't trigger investor alarm bells.

Commoditization is happening, and quickly

The pressure from Chinese open source models is real and grows exponentially every day. OpenRouter reports that these models went from about 2% token usage in 2025 to 60%-70% token usage by mid-2026. During that same period, U.S. frontier models fell from 70% to 30%, proving that in many instances, good enough is just fine, especially when it can drastically lower costs.

The agent boom is playing an outsized role in accelerating this shift, because agents aren't loyal to a brand; they are instead price-sensitive buyers. DeepSeek V4 Flash is priced at $0.09 per million input tokens, compared to $5 for GPT-5.5 -- a not-at-all-insignificant 55x gap. What we've seen here is the fastest-growing AI workload flowing to the cheapest capable model -- which is exactly as it should be.

For many customers, good enough -- and working with a competitor selling at a 55x discount -- is a compelling value proposition, one that the bigger players have no answer to.

Cheap signals, costly signals

The public has largely quit taking industry statements at face value. We've learned that nothing can be believed, and there is always a spin. The trick is figuring out what that is and piecing reality together. That's exactly what we're doing now in the wake of these statements.

Hubinger's public statement that Coxon was correct and that the company does not yet have a plan to address alignment for superintelligence seemed surprising -- and, surprisingly, costly. Just weeks before the IPO filing, I can't imagine that any underwriter on a … 2 trillion-ish book no less … feeling good about that headline out in the wild right before a road show.

Amodei's essay was less costly. Anthropic committed unilaterally only to the first of its three steps: independent evaluation. The expensive steps, however, depend on rivals and governments, and the most important government already said no.

Even the evaluators aren't sold. More than 100 experts signed a letter saying outside oversight is credible only with real independence, deep access and protection from retaliation.

Seeing the news of an agreement this past week between Altman and Amodei to police one another is, quite frankly, somewhat laughable -- kind of like the foxes guarding the hen house.

The audience isn't Washington

The timing makes no sense here if federal rules and oversight were the goal; the reality is that federal regulation of any sort is unlikely to happen any time soon.

A better explanation is the calendar and Anthropic's barreling toward an IPO. An S-1 is expected late September, and a listing is expected in early November. In a prospectus, risk factors become legal representations made under liability.

The more believable story here is that a company that told the world its product might actually end the world needs a credible "and here's how we are managing that" story before they hit the road. Pacing the frontier is that story, and the real audiences are not only institutional investors, but also state legislatures, a mid-term electorate dealing with communities increasingly hostile to the data centers they keep insisting are critical and, of course, enterprise buyers.

Don't lose the present-tense harm

The existential framing of all of this has a strategic side effect: It pulls attention toward what's ahead -- you know, the catastrophes that await us in 2030, and away from the very real harm happening now. Those include the data center energy demand that will no doubt affect climate initiatives, which has been called the new natural-gas buildout; fabricated legal citations used in court filings and other AI slop; and lawsuits alleging insurers used AI models to deny elderly patients care their doctors ordered. 

A regulatory conversation built around superintelligence gives companies a debate where every outcome is hypothetical. We should take care that we don't get lost in the weeds on that front and focus on the rest of the debate as well.

What to watch

What to watch in the weeks ahead? Here's what immediately comes to mind:

The S-1 risk factors

The state of AI expects catastrophic risk to be translated into regulatory, reputational and liability language. How that translation departs from what staff have said publicly is the tell here.

Whether any evaluator agreement meets the evaluators' own conditions

Amodei said Anthropic would give outside evaluators "employee-like access" to monitor and verify the safety of its models. Altman said OpenAI would do the same. It's neat, in theory. But in reality, it's a promise, not a signed agreement.

There are more questions than answers on this front, including who controls the publication of findings (especially those the lab doesn't like), who pays, what's in scope, and what protects the evaluator. If agreements do happen and address each of the foregoing, the commitment is true independence for evaluators, unlimited access (including to unreleased systems), and freedom to publish unfavorable findings and provide real oversights without fear of losing access or retribution. Anything less than that and it's sugar-coated platitudes.

Whether the antitrust waiver gets any traction

If no traction is gained, pacing remains rhetoric and little else.

Nvidia's playbook

Nvidia's incentive is clear: to sell compute to whoever runs models, including enterprises running open-weight models on their own hardware. That would make it both the frontier labs' biggest supplier and most dangerous competitor. Huang has some interesting options here, one of which is providing racks directly to customers that would be on-prem, with capped usage (no more worrying about token costs) and representing a big plus on the data sovereignty front.

In reality, the risk we're discussing can absolutely be real, and the proposals being served up by the players are self-serving. None of this settles the question of whether the danger is real. It's entirely possible that frontier AI poses serious risks and that the companies raising the alarm stand to benefit from the very remedy they're proposing.

For this industry, that seems pretty normal. What we need to collectively do -- as analysts, media professionals, enterprise leaders and enterprise buyers -- is to weigh what each player is asking for against what it would cost them to get it. For example, the White House's refusal costs it nothing politically, as long as nothing goes wrong. Nvidia's insistence on full-speed ahead costs nothing and protects its revenue. Anthropic's commitment to outside evaluators is real, but it's inexpensive. A current employee saying on the record that the company has no plan for its most serious failure mode is expensive, which is why it deserves more weight than Amodei's essay. In short, the more a position costs the person or the organization taking it, the more seriously it should be regarded.

My advice for organizations wondering what to believe and who to trust is simple: don't wait for the federal government or the labs to resolve this. Demand more than safety pledges, insist on evaluation results. Build the internal capability to switch between closed- and open-weight models so that no single vendor's policy or pricing can be overly detrimental. And lastly, let's judge vendors by what they disclose when it is inconvenient: starting with what goes into risk factors of an S-1.

Shelly Kramer is an independent industry analyst, entrepreneur and founder of Kramer&Co. A former founder and operator, she analyzes enterprise technology -- AI and emerging tech, cloud, cybersecurity, IT strategy, customer experience and data -- through the lens of execution, not just capability.