A new decision model launched this month has garnered early interest from developers facing AI cost and governance challenges.
TypeSafe AI, which emerged from stealth Sept 15, released the model, Jev, the same day. Jev is designed for structured decision-making within software, rather than for text generation. It can only return nouls (how true something is), scores (a graded rubric) and choices (multiple-choice question). While less powerful than a full-fledged large language model (LLMs), decision models can make quick determinations that developers can use to guide LLM decision-making.
TypeSafe's founder is Diogo Almeida, a former OpenAI researcher who co-authored a 2022 paper that established reinforcement learning from human feedback as a key method for training LLMs. Almeida described Jev as a System 1 model in a Sept 15 blog post, borrowing a cognitive processing concept from Daniel Kahneman's book, "Thinking, Fast and Slow".
System 1 decisions are reflexive and immediate, said Ned Bellavance, founder and technical educator at Ned in the Cloud.
"Is this shirt blue? Is the pasta cooked? Am I on fire?" Bellevance said.
By contrast, large general-purpose reasoning models are System 2, which are more verbose and non-deterministic.
"System 2 decisions are deliberate and thoughtful," Bellevance said. "What's the best way to approach this situation? How do I really feel about the color blue? What is life?"
Jev's structured outputs could provide a faster and more efficient alternative for developers embedding resource-intensive general-purpose LLMs in their applications to guide decisions, Bellevance said.
"Think of Jev interactions more like a function you'd add to a program or automation routine, and not something that would hold a conversation or write code for you," Bellavance said. "If you need to classify, score, or get a yes/no decision on something, Jev is a good fit."
Jev addresses two common enterprise AI problems, said Torsten Volk, an analyst at Omdia, a division of Informa TechTarget: "One, a lack of deterministic outcomes, and two, token cost."
Torsten Volk
Other popular repos on GitHub have targeted the cost problem caused by unnecessary LLM verbosity using custom Claude skills, such as ponytail and caveman. But Jev's appeal also stems from the fact that it uses a separate model that doesn't require pre-training, as some other decision models do, said Laurie Voss, head of developer relations at Arize, an AI observability platform vendor, in an email to TechTarget.
"For years, companies have been fine-tuning small models into decision models and providing decisions cheaply and quickly," Voss said. "The big jump is that Jev doesn't require any pre-training to do that, just a prompt. That makes it radically easier to adopt, especially for a brand-new app that doesn't have any training data to use. It's also fantastically cheap."
Jev will be priced at $0.042 per million input tokens and $0.00 per output token, according to Almeida's blog post. Claude Fable 5.1, by contrast, costs $10 per million input tokens and $50 per million output tokens, according to a comparison chart in the TypeSafe blog.
There's early evidence that Jev is gaining traction among developers. Vercel, a web application development and hosting platform that is offering Jev free as part of its AI gateway until Sept 25, reported that 13% of its paid AI Gateway users were using Jev within 24 hours of its launch, which represents twice as many teams as any previous model launch, according to a company blog post.
Jev could prompt AI app revisions
Jev isn't necessarily a drop-in replacement for LLMs that existing apps use to make decisions, route and generate responses, Voss said. Jev provides only the decision and routing portions of that workflow, but quickly and cheaply enough that there's an incentive for developers to rearchitect their applications to use it, he said.
"[That means] splitting up your decisions and routing and classification tasks from your text generation, letting LLMs have only that last one, and letting Jev take all the rest," he said. The result is a potentially much faster application that's also happily much cheaper to run."
That speed could make some real-time applications more practical than they would've been with LLMs, according to Bellevance, who cited gaming apps and guardrail generation as two examples.
"Jev is going to be faster and probably better at those System 1-type decisions that appear in automation and programming," Bellavance said.
Jev can run through entire decision loops consisting of multiple questions and updated states 10 times per second, while a traditional LLM would need multiple seconds, Volk said.
It's a super-fast way of getting through very complex workflows that include multiple decision trees and context factors.
Torsten VolkAnalyst, Omdia
"It's hard to say how many seconds, but even if it took only one, Jev would already be 10 times faster," Volk said. "It's a super-fast way of getting through very complex workflows that include multiple decision trees and context factors."
For enterprises, models such as Jev could underpin near real-time sensor data processing in edge microcontrollers, customer service or content analysis workflows.
"Every morning, [I have Fable run through] all the tech news to decide whether it's relevant and no older than 24 hours," Volk said "With Jev, you could have it make the relevant/irrelevant decision instantly. If it's irrelevant, it stops and doesn't consume more tokens. If it's relevant, it hands [the workflow] over to Fable to answer my standard questions."
Trading speed for explainability?
Using a decision model within an AI app could potentially trade explainability for speed, Bellevance said. LLMs produce responses that thoroughly explain their "reasoning," which can increase usage costs, but also enable human audits and evaluations.
"Jev does return a confidence level for its responses, which is nice," he said. "But it does not have a thinking process that chews on a problem for a while. For human-in-the-loop workflows, you're still going to be using an LLM."
For human-in-the-loop workflows, you're still going to be using an LLM.
Ned BellavanceFounder and Technical Educator, Ned in the Cloud
There are also still non-deterministic elements to Jev, Bellevance said – its format is deterministic, but its content isn't. In other words, the model returns a "yes" with a confidence interval, but there's no guarantee of the same answer of level of confidence on a subsequent run.
"I can see this as a distinct category that might displace LLMs in automation workflows," he said. "However, … if you need a deterministic flow, Jev is not going to give you that."
Despite the tradeoffs, early adopters have good reason to jump on this release quickly, Voss said.
"I expect the major model labs will release decision models very, very quickly, and then Jev will have to fight with frontier-level versions of itself and probably compete on price," Voss said. "Once the idea is out in the world, Jev specifically is going to find it doesn't have that much of a moat."
Ben Lutkevich is an award-winning writer for TechTarget.com.
Torsten Volk
