AI cost control
AI cost control: spend caps, quotas and throttling for LLM apps and agents
An alert tells you the money is gone. A cap stops the next request. Leanroute enforces spend and rate limits at the gateway, before any provider is called, for every application and agent that uses it, and shows you where each dollar went.
Hard spend caps
Daily and monthly limits in USD for the whole organization and for each API key. Checked before every request, so once a cap is reached the next request is refused before it reaches any provider. Caps cover your own provider keys (BYOK) too.
How caps work →Application-level quotas
Give each application, team or agent its own API key, and each key its own daily and monthly cap and rate limits. A runaway prototype hits its own ceiling without touching production's budget.
Setting per-key limits →Throttling
Requests per minute and tokens per minute, per organization and per key. Over the limit, requests get a 429 with a header saying which limit fired, so clients can back off instead of guessing.
Throttle headers →Spend less per request
Exact and semantic caching answer repeated prompts at $0. Optional cheaper-model routing swaps to the cheapest live model in the same quality tier; it is off until you turn it on.
Caching →See where the money goes
Usage by model, provider and key; cost per agent run; the cost of each request in the x-gateway-cost-usd response header; the same data from the /platform/v1 API; and email alerts at 80% of a cap.
Cost per agent run →What your code sees
Limits return clear, different errors, so your client can tell "stop" from "slow down":
# Spend cap reached: stop, don't retry
HTTP 402 {"error": {"code": "spend_cap_exceeded", "scope": "key_daily", ...}}
x-gateway-throttle-scope: key_daily
# Rate limit hit: back off and retry
HTTP 429 {"error": {"code": "rate_limit_exceeded", "scope": "key_rpm", ...}}
x-gateway-throttle-scope: key_rpmHonest limits
- A cap stops the next request after you cross it, not one already running. The overshoot is at most the cost of requests in flight at that moment.
- Costs are calculated from catalog prices. On your own provider keys, your provider's invoice is the final word and can differ slightly.
- No budget per agent run yet; caps work per organization and per key.
- Counters live in a fast cache. If that cache is unreachable, caps and rate limits fail open (requests keep flowing rather than all failing) and we get paged.
For agents, start with how to track AI agent costs per run, with LangGraph and n8n examples.
Questions
- How do I set an AI spending limit per application?
- Create one Leanroute API key per application, then set that key's daily and monthly spend caps and its requests-per-minute and tokens-per-minute limits in the dashboard. The organization-wide caps still apply on top.
- Do spend caps work with my own provider keys?
- Yes. Caps are enforced by the gateway on the cost of each request at catalog prices, whether the request uses Leanroute's managed keys or your own provider keys.
- What happens when a cap is reached?
- The next request is refused with HTTP 402 and code spend_cap_exceeded before it reaches any provider. The x-gateway-throttle-scope header says which cap fired. Daily caps reset at midnight UTC, monthly caps at the start of the month.
- Can I cap a single agent run?
- Not yet. Today you can cap the agent's API key per day and per month, and see each run's cost on the Agent runs page. Budgets per run are on the roadmap.
Set your first cap in a minute. New accounts get 100 free requests, no card needed.