LEANROUTE

Blog

Gateway-Run MCP Loop vs. Passthrough: Why Agent Tool Calling Belongs at the AI Gateway

5 min readLeanroute Team

TL;DR: There are three ways to connect Model Context Protocol (MCP) tool servers to an LLM: inside your client code, via an Anthropic-only passthrough, or inside the AI gateway. Running the MCP loop at the gateway converts MCP tools into standard function declarations before dispatch, executes tool calls server-side for up to 10 turns, and returns one OpenAI-shaped completion (marked with x-gateway-mcp-mode: proxy on the response). That means the same MCP servers work on OpenAI, Gemini, DeepSeek, Groq, Qwen, GLM, and Claude without changing client code.

Three ways to wire MCP into an application

When you want an LLM to call tools exposed by an MCP server (such as a calendar, database, internal API, or GitHub server), you have three architectural choices:

Dimension 1. Client-Side MCP Loop 2. Gateway Passthrough (Claude-Only) 3. Gateway-Run MCP Loop (Leanroute)
How it works Your app runs an MCP client, calls tools/list, translates schemas per provider, and runs the while (tool_calls) loop locally. Your app sends mcp_servers to the gateway; the gateway forwards the field verbatim to Anthropic's Messages API. Your app sends mcp_servers on a standard /v1/chat/completions call (stream: false). The gateway discovers tools, dispatches to any model, executes tool calls, and returns the final answer.
Supported models Whichever providers you write custom schema adapters for. Anthropic Claude models only. Any function-calling model in the catalog (GPT, Gemini, DeepSeek, Groq, Qwen, GLM, Kimi, Claude).
Tool-call observability Invisible to the gateway unless you ship separate telemetry. Opaque: the provider executes the calls outside your gateway logs. Tools discovered, tool calls, iterations and MCP server failures are returned in x-gateway-mcp-* response headers, and the request is logged with its full cost.
Guardrails You must run PII/secret redaction in your own process. Apply to the initial prompt only. Gateway guardrails (PII redaction, secret detection, exfiltration checks) apply to the prompt and to the final answer. Tool results passed between turns are not scanned yet.
Server allowlisting Enforced per app repo; easy to drift across teams. Rarely enforced at the org edge. Enforced at the gateway edge via per-org allowlist (/dashboard/mcp) before any connection is opened.

Why we replaced passthrough with a gateway-run proxy loop

We originally shipped MCP support as a passthrough for Claude models. In production, three problems showed up immediately:

  1. You couldn't benchmark or fail over to non-Claude models. Teams building agents wanted to test whether deepseek/deepseek-v4-1-flash ($0.15 / $0.60 per 1M tokens) or openai/gpt-5.6-luna ($0.20 / $1.20 per 1M tokens) could run the exact same MCP tool workflow as anthropic/claude-sonnet-4-6 ($3.00 / $15.00 per 1M tokens) at a fraction of the cost. With passthrough, switching the model string broke the request.
  2. JSON Schema incompatibilities broke cross-provider tool calling. Google Gemini rejects JSON Schemas containing additionalProperties, $ref, allOf, or const, and Gemini 3 requires each function call's thoughtSignature to be echoed back on subsequent turns. OpenAI reasoning models (gpt-5*, o1*) require specific reasoning_effort handling when tools are present.
  3. Zero visibility into runaway loops. When an upstream provider ran the MCP loop internally, our gateway couldn't see how many tool calls failed or cap an agent stuck retrying a broken tool.

How Leanroute's gateway-run MCP loop works under the hood

When a non-streaming request (stream: false) arrives at POST https://api.leanroute.dev/v1/chat/completions carrying an mcp_servers array, the gateway executes a deterministic six-step loop:

  1. Allowlist verification (403 mcp_server_not_allowlisted): Before opening a socket, the gateway checks every URL in mcp_servers against your organization's allowlist at /dashboard/mcp. Unlike a bad model response, a malicious MCP server can exfiltrate prompt data or trigger side effects, so unlisted URLs are rejected at the edge.
  2. Streamable HTTP discovery & namespacing: The gateway connects to each allowlisted MCP server over HTTPS Streamable HTTP and calls tools/list. Every discovered tool is namespaced as serverName__toolName (for example, calendar__list_events and crm__list_events) so multiple servers never collide, then merged with any local tools[] declarations you sent on the request.
  3. Provider-specific schema normalization: The gateway normalizes the merged function declarations for the target model: rewriting const to single-value enum for Gemini, preserving thought_signature across turns, and handling reasoning parameters, so the model never needs native MCP support.
  4. Iterative tool execution (capped at 10 turns): Whenever the model responds with tool_calls, the gateway intercepts the call, strips the serverName__ prefix, executes the tool against the matching MCP server with your authorization_token (which is never logged or echoed), and feeds the tool output back into the next model turn.
  5. Freshness guarantee (cache bypass): Because the output of an MCP tool call depends on live external state ("What meetings do I have today?"), requests carrying mcp_servers automatically bypass both the exact-match and semantic caches on read and write.
  6. Single OpenAI-shaped response + full header telemetry: Once the model produces a final answer (or hits the 10-iteration ceiling), the gateway returns one standard OpenAI completion. The usage object sums input and output tokens across every iteration so the token count you see matches your exact bill.

Try it in one cURL call on GPT, DeepSeek, or Gemini

You don't need an MCP client SDK in your application. Point a standard /v1/chat/completions call at any function-calling model in the Model Catalog:

curl https://api.leanroute.dev/v1/chat/completions \
  -H "Authorization: Bearer $LEANROUTE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek/deepseek-v4-1-flash",
    "stream": false,
    "messages": [
      { "role": "user", "content": "Check my calendar MCP server and summarise tomorrow'\''s schedule." }
    ],
    "mcp_servers": [
      {
        "type": "url",
        "url": "https://calendar.example.com/mcp",
        "name": "calendar",
        "authorization_token": "eyJhbGciOi..."
      }
    ],
    "max_tokens": 2048
  }'

mcp_servers works on non-streaming requests only. Sending it with "stream": true returns a 400 that says so.

Next Steps

MCPModel Context ProtocolAI GatewayLLM GatewayAgent ArchitectureTool Calling
← All posts