What is LLM routing, and when does it actually save money

"LLM routing" gets used loosely enough that it's worth defining plainly before arguing about whether it works. A model router sits between your application and the model providers, looks at each request, and sends it to whichever model in a set of candidates it judges can handle that request, usually with cost as the deciding factor when several models could plausibly do the job. The pitch is straightforward: not every request needs your most expensive model, so why pay for it every time.

That pitch is correct as far as it goes. The part worth understanding, because it changes how much you should trust any specific router's claims, is what the router actually has to work with, and how confidently it can act on it.

Why blended pricing varies so much

Model pricing spans a genuinely enormous range. Frontier models can be priced tens to over a hundred times higher, per million tokens, than smaller models from the same provider or a competitor, and the gap isn't a rounding difference, it's the difference between a request costing a fraction of a cent and costing several cents. Multiply that across millions of requests and the blended cost of "always call the strongest model" versus "call the cheapest model that works" is often the single biggest lever available on an LLM bill, bigger than prompt optimization, bigger than caching, bigger than almost anything else you'd tune first.

That's the entire economic case for routing existing in the first place. If pricing were flat across models, routing would just be a reliability feature (pick whichever provider is up) and nobody would be writing about it as a cost strategy.

What a router can infer from the prompt, and what it can't

Here's the part that gets skipped in most routing pitches: deciding whether a cheap model can handle a given request is itself a hard prediction problem, and it's hard specifically because the router usually only has the prompt text to go on before generation happens.

Published research on prompt-level, pre-generation routing (deciding which model to call based on features of the prompt alone, before any model has actually answered) consistently finds this harder than intuition suggests. Task difficulty doesn't reliably show up as a legible signal in prompt text. A short, simple-looking question can require real reasoning; a long, complex-looking prompt can be mechanical extraction a small model handles fine. Surface features like length, vocabulary, or question type are only loosely correlated with whether a cheap model will actually get the answer right, which is why routers trained primarily on those features tend to either play it safe (route up more often than necessary, eroding the savings) or get it wrong in the direction that matters most: sending a request that needed a strong model to a weak one, silently.

None of this means pre-generation routing is useless. Trained classifiers that route based on prompt features do measurably better than random assignment, and for a lot of traffic that's a real, usable improvement. It means the honest framing is "this improves your odds," not "this reliably knows what your prompt needs," and any router that implies the second thing is overselling what prompt-level inference can actually do.

Why post-generation verification changes the calculus

This is the actual hinge point, and it's the reason routing-without-checking and routing-with-checking are different products even when they share the word "router."

If your only signal is a pre-generation guess about the prompt, routing down is a bet. You send the request to the cheap model, you don't find out whether it worked, and the savings figure you report is real only if the guess was right often enough. When it's wrong, the cost isn't just "we should have used the expensive model." It's a bad answer that reached a user or a downstream system with nobody having checked it, which is a much larger cost than the few cents you saved on that one call.

Add a check after generation and the whole risk profile changes. Now the router doesn't have to correctly predict difficulty in advance; it just has to route down aggressively and be willing to catch the cases where that was wrong. A schema check catches malformed output for free. A groundedness check, run against context you actually supplied, catches claims the model made that the source material doesn't support. Heavier checks, cross-model comparison or resampling, catch harder-to-detect failures on the requests that deserve the extra cost of running them. Any request that fails its check escalates automatically to a stronger model, so the bad answer never actually ships. You end up with a system that can safely route down further and more often than pure guesswork would ever justify, because the failure mode isn't "silently wrong," it's "caught and retried."

This is also why a routing project's real savings depend on which of these two things it's actually doing. "We route to cheap models based on prompt features" and "we route to cheap models and verify what comes back" produce very different reliability profiles even when the marketing copy for both says "save money on your LLM bill."

The practical version

If you're evaluating whether routing will actually save you money without a hidden reliability cost, the question to ask any router, including this one, is simple: what happens when the cheap model gets it wrong? If the answer is "we don't check, so we don't know," that's routing on faith. If the answer involves a verification step and an escalation path, that's routing you can actually rely on.

llm11 works the second way: a triage layer picks a candidate model per request, and a verification ladder underneath checks the answer before it goes out, escalating to a stronger model when a check fails. The router page covers how the triage decision itself works, and the verification page covers what the checks catch and, just as importantly, what they don't claim to catch.