Agent-native AI gateway
What is an agent-native AI gateway?
An AI gateway sits between your software and the models it calls: one endpoint, routing, failover, caching, spend limits. That was built for apps, where one user action is one model call. Agents are different. One task becomes dozens of model calls and tool calls, and the questions change: what did that run cost, what did the agent touch, and was it allowed to?
An agent-native AI gateway keeps everything an AI gateway does and adds what agents need. Here is the checklist we hold ourselves to, with a test you can apply to any gateway, including ours.
1 of 5
It understands runs, not just requests
An agent turns one task into many model and tool calls. A request log tells you what each call cost; it can't tell you what the task cost, or which of your agents is expensive.
The test: Can you see, per agent run, how many calls it made, which models it used, which tools it called and what it cost in total?
In Leanroute: Send X-Agent-Name and X-Agent-Session-Id; every request with the same session id becomes one run, with cost, models and tools, in the dashboard and through GET /platform/v1/runs. Streaming included. Agent runs →2 of 5
Tool calls are first-class traffic
For an agent, a tool call matters as much as a model call. A gateway that only sees prompts and completions misses half of what the agent did.
The test: Does the gateway record tool calls by name, both the ones it runs and the ones the model asks the agent to run? Can it run MCP tools for models other than Claude?
In Leanroute: The gateway runs the MCP tool loop itself, so MCP tools work on GPT, Gemini, DeepSeek and others. Tool calls the model asks your agent to run are recorded too. Names and timings only; arguments and results are never stored. MCP on any model →3 of 5
It controls what agents can reach
A bad model answer can be retried. A bad tool call can send data somewhere or change a system. The gateway is the natural place to say which tools are allowed.
The test: Is there an allowlist the gateway enforces before any tool runs, and can an agent find out what it is allowed to use?
In Leanroute: MCP servers must be on your organization's allowlist or the request is refused. Agents can list the servers they may use at GET /v1/mcp/servers. Discovery →4 of 5
Agents can operate it
If an agent is doing the work, it should be able to answer "how much have we spent?" or "what did that run do?" without a person opening a dashboard.
The test: Is there an API and an MCP server with scoped credentials, so an agent can read usage and runs without being able to do anything dangerous?
In Leanroute: A hosted MCP server and the /platform/v1 API expose usage, runs, pricing and key management under a separate admin key class. Top-ups and provider keys stay dashboard-only. MCP server →5 of 5
It fits the frameworks agents are built with
Agents are built in LangGraph, the OpenAI Agents SDK, the Vercel AI SDK, n8n or Claude Code. A gateway that needs its own SDK will not be adopted.
The test: Does it work by changing a base URL, and keep state out of the gateway so the agent stays in charge of its own memory?
In Leanroute: OpenAI-compatible and Anthropic-compatible endpoints, two optional headers, no gateway-side session state. Claude Code sessions show up as runs with no setup. Framework recipes →
Still an AI gateway underneath
Agents need the basics as much as apps do, more so given how many calls they make. An agent-native gateway should not trade any of them away:
- One OpenAI-compatible API across many providers, with failover when one is down
- Caching, so repeated calls cost nothing
- Hard spend caps that stop a runaway loop before it reaches the provider
- Guardrails on what goes in and comes out
- Clear data handling: what is stored, where, and for how long
Where it goes next
Not shipped yet in Leanroute, and the natural next steps for the category:
- Per-key allowlists, so one agent can use a tool another cannot
- Budgets per run, with a hard stop
- Policy over the agent's own function tools, not only MCP servers
- Short-lived credentials for subagents
How gateways compare
Several gateways cover parts of this list: some group requests by session, some run MCP tools, some track spend per user or tag. The comparison table shows who does what, checked against each vendor's documentation.
Questions
- What is an agent-native AI gateway?
- An AI gateway built for software agents as well as apps. Besides routing model calls, it groups an agent's calls into runs, treats tool calls as first-class traffic, controls which tools agents can reach, can be operated by agents themselves, and works with existing agent frameworks.
- How is it different from a regular AI gateway?
- A regular AI gateway sees independent requests: a model, tokens, a cost. An agent-native gateway also sees the run those requests belong to and the tools called along the way, so it can answer what a task cost and what an agent actually did.
- Is it an agent framework or an observability tool?
- Neither. Agents are still built in your framework, and their memory and state stay there. The gateway sits between agents and the models and tools they call, enforcing rules and recording what happened. You can still send traces to an observability tool.
- Does Leanroute store agent state?
- No. The agent sends a session id with each request and Leanroute groups by it at query time. There is no session to create or close.
Try it with two headers. New accounts get 100 free requests on low-cost models, no card needed.