↓ Skip to main content
Expert LinkedIn
AI AI Engineering Applied AI

Your Next AI Feature Shouldn't Write a Word: The Quiet Rise of Decision Models

10 min read
Alex Winters
Alex Winters Prompting Specialist & Writer

Here is a design decision most teams made by accident: they pointed a chatbot at a judgment.

When a support system needs to know whether a message is really asking for a refund, when a pipeline needs to label a document, when an agent needs to pick one of six next steps, the default is still to prompt a chat model and read the answer out of the prose it returns. It works, which is why we kept doing it. It also means paying for output tokens, waiting on paragraphs, and then asking a human to parse a sentence that was secretly a yes or a no.

That default started to crack last month. On September 21, a startup called TypeSafe AI shipped Jev, which it pitched as the first of a new category: “System One models”. You send it text or a semi-structured record, and instead of writing back, it returns numbers: a confidence score for a yes/no question, a probability distribution across a set of choices, or a float along a score range. TypeSafe called it “a frontier-intelligence function call: unstructured state in, typed probabilistic decisions out.”

A precision brass decision dial glowing amber on a dark machined panel, while an enormous tangled scroll of printed prose spills from a slot below and trails unread into shadow.
The valuable output is one calibrated setting — not another page of words nobody will read closely.

Two weeks later, the category stopped being a novelty. OpenAI released a Jev-style Decisions API, with a model the community promptly plugged into existing tooling, charging for input and not output (Simon Willison, October 6). And in the same week, Papers With Code filled up with the academic version of the same idea: “Jeva: System One Models from First Principles,” “VideoJev,” and a decision model explicitly evaluated on calibration. Four weeks, one startup to one big-lab endpoint to a research subfield. That is usually the signature of a component class forming, not a demo.

In four weeks, a judgment became a typed object
#

The reason this matters is not that the models are clever. It is the shape of the output.

A decision model doesn’t hand you an artifact a human must read. It hands you a value a system can gate on. That single change moves a task out of “generation” and into “measurement.” A yes/no question comes back as a number between 0 and 1. A choice question comes back as a probability distribution across the options you supplied. A score question comes back as a point along a scale you described. The same record can carry many questions at once, and they evaluate in parallel, so asking twenty questions costs about as much wall-clock time as asking one.

That is why the use cases that keep coming up are classification-shaped and high-volume: spam and abuse triage, intent routing, content labeling, claim and eligibility checks, prioritization and ranking, retrieval reranking, tool selection inside an agent loop. Notice how many of those are things teams currently do by prompting a chat model, parsing its sentence, and throwing the sentence away.

Why this is cheaper than it looks
#

The economics are the tell. Jev charges only for input — about $0.042 per million tokens, with output free — which is cheaper than a budget chat model, and orders of magnitude cheaper than a frontier one. OpenAI’s decision endpoint follows the same shape.

That price structure is not a stunt. It is what happens when a vendor decides the output is a decision rather than a document. And it changes the arithmetic that has quietly governed AI products all year. In The Token Capital Trap I argued that output tokens are where budgets go to die, because we keep buying prose for tasks that don’t need it. Decision models are the obvious payoff: they make the boring ninety percent of a workflow — the routing, the triage, the “does this qualify?” — cheap enough to run on every single item, and cacheable enough to run once.

That last point is underrated. A decision is one of the few AI outputs you can meaningfully cache: the same record plus the same question should yield the same number, which means a decision layer behaves like infrastructure instead of a slot machine. This is also where it meets the rest of this year’s lesson in recovery and runtime design: cheap, stateless, re-runnable judgments are exactly the primitives you want inside an agent loop that has to fail gracefully.

The uncomfortable part: black boxes are back
#

Now the part that should slow you down.

The convenience of a decision model is that it hands you a number. The danger is the same fact. There is no rationale to read. Ask a chat model why it flagged a comment and you can at least ask for a reason, even if the reason is unreliable. Ask a decision model and you get a float. Willison’s own probe is the cautionary tale: scoring every Bay Area city on a “good city?” question put Cupertino on top and East Palo Alto at the bottom, and no explanation came with it.

That is not a reason to avoid the category. It is a reason to change what you test. If a single number is going to decide whether a claim is paid, a comment is hidden, or a candidate moves forward, then evals stop being a quality check and become the product. A prose model fails visibly — someone reads a weird sentence. A decision model fails invisibly, in a number that looked confident. So the eval work shifts from “did the output look good?” to harder questions: is the probability calibrated, does it survive when you reorder the options, does it stay stable when the record is empty or contradictory, and can you show it isn’t quietly encoding the wrong pattern? Decision models make the craft of measurement the whole job. And the demand for proof is arriving from outside the team, too: this month Anthropic began embedding independent evaluators with employee-like access who can publish their own analysis of a model’s behavior (Anthropic, October 15). “Provable” is becoming an external expectation, not just an internal virtue.

What actually changes in your stack
#

If this holds, the applied skill for next year is a routing skill. Stop treating every AI step as generation; start labeling each one expression or judgment.

Expression is where you genuinely need language — drafting a reply, summarizing a messy thread, explaining a decision to a customer. Judgment is where you need a state in and a decision out: is this abuse, does this qualify, which queue, how urgent, yes or no. Expression stays with generative models and humans in the loop. Judgment wants a typed, cheap, cacheable component with a calibration test attached.

A short version of the checklist I’d put on a wall:

  1. Is the answer secretly a value? If the prose you parse collapses to yes/no, a label, or a rank, it was always a decision.
  2. What does a wrong number cost? Cheap and reversible, run it everywhere. Expensive or irreversible, cap the action behind a human or a second check.
  3. Can you interrogate it? If you cannot test the number for calibration and bias, you are not shipping a feature. You are shipping a liability with a confidence score.
  4. Can you cache it? Same record, same question, same answer. If you can, you just turned a per-request cost into infrastructure.

Judgment is about to become the bottleneck
#

There is a second reason this is arriving now, and it comes from the other end of the industry. Anthropic disclosed this month that it had to replace Git itself because agent-scale development broke human-scale tooling: merged changes rose from about 2,500 a day in May to more than 30,000 a day by September, with roughly 25 million code checkouts a day for CI alone (Anthropic Engineering, October 13). When that much change flows, you cannot read all of it. So the team built semantic approval rules that separate a comment-only edit from a sensitive logic change, so human attention lands only on the consequential ones. That is a decision model in spirit: classify the change, route the scrutiny.

The same reframe is showing up in review generally. Asked how the team behind Claude Code reviews code, the answer was not “did a human read it?” but “what could go wrong, and which check — automated or human — is the right one to catch it?” (Simon Willison, October 16). And as agents multiply, the guardrail everyone suddenly wants is a hard budget cap: a spend limit that stops a runaway loop instead of emailing you about it at midnight (Simon Willison, October 3). More autonomy is forcing more classification — what is consequential, what is reversible, what should never run unattended. Those are judgments. They want a decision layer.

The debate worth having
#

There is a real disagreement in the field about where reliability lives, and it is worth naming honestly.

One camp says the model is not the problem — the harness around it is. Give a general model better tools, better scaffolding, and better recovery, and it will handle decisions inside its flow. The research wave leans this way too, with recursive “harness” papers and agents trained to reflect and recover from noisy feedback. If you believe this, decision models are an optimization, not a re-architecture.

The other camp says some steps should never touch a generative model at all, because for those steps prose is not a feature, it is overhead you have to verify. My colleague Emily Chen made the human version of this case this week: the “verification tax” is the invisible bill that lands on careful and junior workers who have to check what the machine wrote. Decision models do not remove judgment from the system. They move it to a place where it is a single testable value instead of an under-reviewed paragraph. Where Emily’s argument and mine meet is simple: the cheapest way to stop paying the verification tax on judgment-shaped work is to stop buying prose for it.

I land between the two camps, and slightly against my own enthusiasm. The harness matters, and I would not rip out a working generative pipeline to chase a trend. But the harness argument does not explain why we let a system that is bad at calibrated numbers make so many decisions in the first place. Most teams are not choosing between two architectures. They are defaulting to the wrong output type because it was the only one available.

That default is now a choice. And once a judgment becomes a number, it becomes something you can version, rate, cache, and prove — which is exactly the standard the rest of this quarter is being judged against. Deadlines got faster this month, and praise outran compensation. Calibration is turning into the currency of late 2026, for people and for models alike.

So here is the line I keep coming back to. A judgment that becomes a number you can prove travels. A paragraph you have to argue about does not.

Building something on top of a decision model, or trying to prove one isn’t biased? I’d like to hear where the number stops being trustworthy in your stack.

Email me at alex.winters@tlnw.uk

Data infographic contrasting generative models that write paragraphs with decision models that return a typed number, showing Jev and OpenAI input-only pricing, three decision types, and Anthropic's agent-scale code volume.
The decision layer costs less than the paragraph it replaces — and hides its reasoning

References
#

AI Content Notice

This article was created using artificial intelligence technology. Whenever possible, we include references and sources to support the information presented. Readers are encouraged to consult these sources for further information. While we strive for accuracy and provide valuable insights, readers should independently verify information and use their own judgment when making business decisions. The content may not reflect real-time market conditions or personal circumstances.

Related Articles

Tags

#decision models #system one #ai evals #model routing #ai bias