LLM routing with Leanroute: tiers, policies and failover

Most teams start with one model from one provider. Then a cheaper model appears that is good enough for half the traffic, a provider has a bad afternoon, and someone in compliance asks which vendors your prompts actually reach. Routing is the answer to all three, and it is easier to reason about when you can see exactly what the gateway decided and why.

This post walks through how routing works in Leanroute, what you can configure, and how to verify each decision from the response.

The default: you get the model you asked for

Leanroute is OpenAI-compatible. Point your SDK at https://api.leanroute.dev/v1, use your Leanroute key, and name a model with the provider/model form:

curl https://api.leanroute.dev/v1/chat/completions \
  -H "Authorization: Bearer $LEANROUTE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-4-6",
    "messages": [{"role": "user", "content": "Summarise this ticket in two lines."}]
  }'

With no routing settings changed, that request goes to Anthropic and is served by Claude Sonnet 4.6. Nothing is swapped. Bare names like claude-sonnet-4-6 or gpt-... also work, but the prefixed form is unambiguous, and some providers' model names collide, so we recommend it.

If you call a model the provider has retired, Leanroute rewrites it to the documented replacement so your request does not break, and tells you with an x-gateway-model-deprecated header so you can update your code.

Quality tiers

Every model in the catalog sits in one of three tiers:

flagship: general-purpose frontier models.
cheap_fast: smaller, faster models for high-volume work like classification, extraction and short summaries.
premium_reasoning: models built for long reasoning chains.

The tier for each model is published on the tiers page, along with the public benchmarks that place it there. Tiers are what make a swap safe to reason about: Leanroute only ever swaps a model for another model in the same tier.

Cheaper-model swaps

Turn on Cheaper-model swaps under Dashboard, Settings, Routing. With it on, a request for a model is served by the cheapest model in the same tier among the providers you can reach. A request for a flagship model may be served by a cheaper flagship-tier model from another provider.

Three kinds of request are never swapped, even with the setting on:

Requests with tools. Tool-call formats differ enough between providers that a silent swap could break your code.
Requests with mcp_servers. The model running a tool loop should be the one you chose.
Requests with images. Vision models read documents and screenshots differently, so swapping them changes answers in ways you did not ask for.

To pin a single request to the exact model you named, even with swaps on, send:

x-gateway-routing: explicit

That is useful for benchmarking, evals, or any call where you need one model's exact behaviour.

Routing policy: which providers your prompts can reach

The same Settings page has three controls that shape where swaps and failover can go:

Allowed providers. A whitelist. When set, only these providers are considered. Use it for "we have contracts with these three vendors and nobody else".
Blocked providers. A blacklist that always wins, even over the whitelist.
Preferred region. Global, China or India. Candidates in the preferred region get a small cost advantage in the picker, enough to break near ties without overriding a clearly cheaper option. It is a preference, not a hard filter.

These rules apply to swaps and failover. The model you explicitly name in a request is still the one that is called.

Failover

If the provider serving your request returns a 5xx, a 429 from its own rate limits, a timeout, or an auth error on its key, Leanroute retries on the cheapest same-tier model from another provider your policy allows. Your client sees one successful response.

Failover does not run for:

4xx request errors like a malformed body. Another provider would return the same error, and retrying would just cost money.
Requests with tools, for the same reason as swaps.
x-gateway-routing: explicit requests.
Streams that have already started. If the provider fails before the first byte, Leanroute fails over. Once tokens are flowing, it cannot cleanly switch mid-answer, so the stream ends with an error instead.
Your own keys mixed with ours. On the credits plan, a request made with your own provider key never fails over onto Leanroute's keys, because that would bill your balance for traffic you meant to send through your own account. On a BYOK plan, failover moves only between providers you hold keys for.

Verify every decision

Routing you cannot audit is routing you cannot trust. Every response carries headers that say what happened:

Header	What it tells you
`x-gateway-provider`	The provider that actually answered
`x-gateway-routed-from`	The model you asked for, when a swap happened
`x-gateway-swap-reason`	Why it was swapped, for example `cheapest-in-tier`
`x-gateway-benchmark-basis`	The benchmarks that place the served model in its tier
`x-gateway-failover`, `x-gateway-failover-from`	That failover ran, and which provider failed
`x-gateway-cost-usd`	What this request cost

In Python with the OpenAI SDK:

from openai import OpenAI

client = OpenAI(base_url="https://api.leanroute.dev/v1", api_key=LEANROUTE_API_KEY)

raw = client.chat.completions.with_raw_response.create(
    model="anthropic/claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Summarise this ticket in two lines."}],
)
print(raw.headers.get("x-gateway-provider"), raw.headers.get("x-gateway-routed-from"))
completion = raw.parse()

The same information lands in your request logs in the dashboard, so you can see after the fact how much traffic was swapped and how often failover saved a request. The full list is on the headers reference.

A sensible starting setup

Start with swaps off and send real traffic for a few days. Watch cost per model in the usage dashboard.
Turn swaps on and compare quality on a sample of responses. Pin anything sensitive with x-gateway-routing: explicit.
Add an allowed providers list if procurement or data residency limits which vendors you can use.
Leave failover to do its job, and alert on x-gateway-failover in your own monitoring so you know when a provider is having a bad day.

You can try requests without an account in the playground, or read the API docs to wire it into your app.