How a request actually gets routed with Jev.

Stage one: building the pool

Before triage runs at all, the request’s candidate pool gets resolved: an explicit list of model ids on the request, a named price band like llm11-balanced, or the project’s own default pool if neither is supplied. A pinned single model id skips routing entirely and the pool is just that one model. Every other case hands triage a sorted, cheapest first list of real catalogue ids, priced against the live upstream catalogue at request time.

This matters because Jev is never asked to invent a model. It is asked to pick from a specific, priced, already legitimate set of options, which is a much narrower and much more answerable question than “what should answer this” in the abstract.

Stage two: one parallel pass, several typed questions

The request’s messages become Jev’s state, and four typed questions go out alongside it in a single call: what kind of task this is, which pool member is the cheapest one that will still answer it well, whether a wrong or unsupported answer here would cause real harm, and whether the apparent stakes and difficulty warrant checking the answer against a second independent model. Jev answers all four in around 70 to 500ms, with a calibrated confidence attached to the choices that matter.

The model pick becomes the request’s primary completion target. The harm question, answered confidently rather than as a shrug near 0.5, raises the request’s criticality. The cross-check question, answered confidently, adds the cross model rung to the verification plan. None of this is a second round trip: it is one pass producing every decision the rest of the pipeline needs.

Why calibration matters more here than raw accuracy

Routing is a repeated decision problem, not a one shot answer. Thousands of requests a day each get a confidence number, and that number is what decides whether a request gets the cheap default path or an extra, paid verification rung. If the number is merely correct on average but not trustworthy at the extremes, the whole mechanism degrades into noise: some easy requests get expensive checks they did not need, and some genuinely uncertain ones sail through with nothing. A model that is simply accurate does not fix that. A model whose 0.05 really means unlikely and whose 0.95 really means likely does, because the gate can be set on the number and trusted to hold.

That is the property independent testing found in Jev: items it scored under 0.1 behaved like low-confidence cases, items above 0.9 behaved like high-confidence ones. It is why the “needs cross check” answer above is read as confident yes only past 0.75, not past 0.5. A shrug should not spend your money on a second model call.

Stage three: generation, then the verification ladder

The picked model answers the request. The verification ladder then runs whichever rungs triage flagged, cheapest first: a schema check always, a groundedness check against any context the caller supplied, a cross-model second opinion when flagged, and self-consistency resampling reserved for the narrow case of high criticality combined with triage’s own confidence sitting in an uncertain middle band. Each rung can pass the request through or stop it there.

Stage four: escalation, automatic and bounded

When a rung fails, whichever one it is, the request escalates once, automatically, to the strongest remaining member of the same pool: the most expensive model that is not the one that just failed. That stronger model answers again, and gets re-checked with the schema rung and, if context was supplied, the groundedness rung. It does not escalate a second time. An escalation loop is worse for whoever is waiting on an answer than accepting the strong model’s response with an honestly recorded failed first attempt, so the ladder stops there by design.

A single-model pool, a pinned request, has nowhere stronger to escalate to, so it answers again on the same model rather than failing the request outright.

On the heuristic fallback path

If Jev is not configured on a deployment, unavailable, slow, or over an affordability threshold, requests fall through to a built-in heuristic backend instead. It needs no network call, costs nothing, and never throws. It reads the request text for a small set of surface signals, code syntax, reasoning language, a handful of high-stakes keywords like contract, diagnosis, or compliance, and picks a pool position from those: cheapest, middle, or most expensive by blended price.

Its confidence is capped well below what Jev reports, at levels that reflect being a fixed set of rules rather than a calibrated model, and when it is unsure it escalates criticality rather than guessing low. The practical effect is a more conservative pool position and more frequent cross-model checks: the fallback trades some routing precision for staying safely wrong in the expensive direction rather than the cheap one.

What the receipt shows

Every request’s receipt records the pipeline’s decisions, not just its output: which backend triaged it, at what confidence, which pool it chose from and where that pool came from, which rungs ran and whether each passed, whether the request escalated and why, and the most expensive model in the offered pool as the baseline the savings figure is measured against.