MCP tool calling on any model, not just Claude

MCP has a reputation as a Claude feature, largely because Anthropic's Messages API was the first major chat API to accept an mcp_servers field. The simplest way for a gateway to support MCP is to forward that field to Anthropic and step out of the way.

That works. It also means MCP only ever runs on Claude, and it means the gateway cannot see a single tool call.

We shipped it that way in August. We replaced it this week.

We're not the only gateway to make this move. LiteLLM's self-hosted proxy runs MCP tools on any backend, and Portkey has gateway-side MCP execution in beta. What follows is how we did it on the plain OpenAI chat-completions API, and what running the loop ourselves exposed.

What changed

Leanroute now runs the MCP agent loop itself:

1. Connect to each MCP server you listed and ask it what tools it has
2. Namespace them, so two servers can both expose a search tool without colliding
3. Inject them as ordinary OpenAI-style function declarations, alongside any tools you declared yourself
4. Dispatch to whichever model you asked for
5. Intercept the tool calls the model makes, execute them against the right server, feed the results back
6. Loop until the model stops asking for tools, then return one response

Step three is the whole trick. Once MCP tools are plain function declarations, every provider with function calling supports MCP: GPT, Gemini, DeepSeek, Groq, Grok, Claude. There is no per-provider MCP integration to build or wait for.

The request shape didn't change:

{
  "model": "openai/gpt-5.6-luna",
  "messages": [{ "role": "user", "content": "What upcoming meetings do I have this week?" }],
  "mcp_servers": [
    {
      "type": "url",
      "url": "https://your-mcp-server.example.com/mcp",
      "name": "calendar",
      "authorization_token": "eyJhbGciOi..."
    }
  ],
  "max_tokens": 2048
}

Note the model. That's the difference.

The part that surprised us

Running the loop ourselves means we see every tool call. Which server it hit, what it cost, how long it took, whether it failed. A forwarder sees none of that. It hands mcp_servers to Anthropic and gets back a finished answer with the tool calls already buried inside it.

We expected that to matter for observability. It mattered more than that: it surfaced three bugs that had been in production for weeks, on paths that had nothing to do with MCP.

Gemini's JSON Schema subset. Gemini's function declarations accept an OpenAPI 3.0 subset, not full JSON Schema. A tool schema containing additionalProperties, $ref, allOf or const got a 400 from Google while the identical schema worked on OpenAI and Anthropic. MCP servers commonly emit additionalProperties: false, so the moment we started injecting real MCP tools we walked straight into it. Any customer passing those keywords had been hitting it for months.

Gemini 3 thought signatures. Gemini 3.x attaches an opaque thoughtSignature to every function call it makes, and requires it echoed back on the next turn. We were discarding it during translation. The symptom is diabolical: turn one always succeeds, turn two always fails with a 400. Easy to misread as a bug in your tool-result handling, which is where we looked first, three times.

OpenAI reasoning models plus tools. Reasoning models carry a server-side default reasoning_effort, and OpenAI rejects any /v1/chat/completions request combining it with function tools unless it is explicitly none. We never set the field. The request still failed.

All three were pre-existing, all three were customer-facing, and all three were invisible until something exercised tool calling harder than our test suite did. An agent loop turns out to be a very effective fuzzer for your own tool-calling path.

What it costs you to run the loop

Honest accounting, because this is the part gateways tend to skip.

A tool-using request is several model calls, not one. The tool results become input tokens on the following turn, so a chatty tool inflates your prompt fast. We aggregate usage across every hop and return the total, so the number in usage is the number you are billed. No surprises at the end of the month.

There is a hard ceiling of ten model round trips per request. If a model gets stuck retrying a failing tool, you get a response and a header saying the cap was hit, rather than an unbounded bill.

Guardrails, because MCP servers are not neutral

An MCP server can execute actions on your behalf. Unlike a bad model, which is recoverable by retrying elsewhere, a bad MCP server can exfiltrate data or write to systems.

So mcp_servers URLs must be on your organisation's allowlist before we will connect to them. Unlisted URLs return 403 mcp_server_not_allowlisted with the offending URLs echoed back. Add them at /dashboard/mcp.

Auth tokens you supply go to your server and nowhere else. They are never logged and never placed in a response header.

What is not supported yet

Streaming. Intercepting tool calls mid-stream means buffering the upstream SSE, detecting the tool turn, executing, then splicing a second upstream response into the same client connection. Real work, and we would rather do it when someone needs it than guess at the ergonomics now. stream: true with mcp_servers returns 400 mcp_streaming_unsupported on every provider. Set stream: false and we run the loop for you, which is what most agent code wants from a tool turn anyway.

Per-tool allowlisting is also absent. Allowlisting is at the server level today.

Try it

Full reference at /docs/mcp. The shortest version: add mcp_servers to a non-streaming /v1/chat/completions request, point model at anything with function calling, and read the x-gateway-mcp-* response headers to see exactly what the loop did.

If you point it at a model that isn't Claude and it works, that's the feature.