LangChain + Leanroute

1. Get a gateway key

From /dashboard/keys create a key labeled "langchain". Copy the gw_live_* value — it's shown only once.

2. Python: chat

pip install langchain-openai if you don't already have it. Then:

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="anthropic/claude-sonnet-4-6",   # or openai/gpt-4o, deepseek/deepseek-v4-flash, etc.
    api_key="gw_live_YOUR_KEY",
    base_url="https://api.leanroute.dev/v1",
)

print(llm.invoke("Explain retrieval-augmented generation in one sentence.").content)

The model string uses our canonical provider/model form. Full list at /dashboard/models.

3. Python: streaming

for chunk in llm.stream("Write a haiku about caching."):
    print(chunk.content, end="", flush=True)

Server-sent events are forwarded from the upstream provider chunk-by-chunk with no intermediate buffering — your first token latency matches going direct.

4. Python: embeddings

from langchain_openai import OpenAIEmbeddings

embed = OpenAIEmbeddings(
    model="openai/text-embedding-3-small",
    api_key="gw_live_YOUR_KEY",
    base_url="https://api.leanroute.dev/v1",
)

vec = embed.embed_query("what is prompt caching?")
print(len(vec))  # 1536

Use the same key. Any LangChain vector-store integration (Chroma, Pinecone, pgvector, Weaviate, etc.) accepts an embedding_function argument — pass this embed and your entire indexing + query flow now runs through Leanroute.

5. JavaScript / TypeScript

npm install @langchain/openai. The JS package uses configuration.baseURL (nested) instead of the flat base_url Python uses.

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
  model: "anthropic/claude-sonnet-4-6",
  apiKey: "gw_live_YOUR_KEY",
  configuration: {
    baseURL: "https://api.leanroute.dev/v1",
  },
});

const res = await llm.invoke("Explain retrieval-augmented generation in one sentence.");
console.log(res.content);

6. Agents + tool calling

LangChain agents (React, tool-calling, structured-output) work end-to-end because Leanroute forwards the tools and tool_calls fields to the upstream unchanged. Any model that supports tool calling upstream supports it via us — every flagship SKU today does.

from langchain.agents import create_react_agent, AgentExecutor
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI

@tool
def get_weather(city: str) -> str:
    """Return the current temperature for a city."""
    return f"{city}: 22°C"

llm = ChatOpenAI(
    model="anthropic/claude-sonnet-4-6",
    api_key="gw_live_YOUR_KEY",
    base_url="https://api.leanroute.dev/v1",
)

# Tool calling works because Leanroute proxies function_call / tool_calls
# transparently. Model must support tools (all flagship models do).
agent = create_react_agent(llm, tools=[get_weather])
executor = AgentExecutor(agent=agent, tools=[get_weather], verbose=True)
print(executor.invoke({"input": "What's the weather in Singapore?"}))

Optional: cross-provider fallback

Enable the cross-provider failover toggle in /dashboard/settings and any 5xx from your chosen provider (e.g. Anthropic 529 overload) will retry on the cheapest same-tier alternative. Your LangChain code doesn't change; the retry is transparent from the SDK's perspective. Response header x-gateway-routing tells you which provider actually served the request.

Troubleshooting

• 401 unauthenticated: the key was rotated or revoked. Issue a new one at /dashboard/keys.
• 400 unknown_model: the model name doesn't match our canonical list. The provider prefix is required (use anthropic/claude-sonnet-4-6, not claude-sonnet-4-6).
• Tool calls not firing: confirm the model supports tools upstream (all flagship models do; some cheap_fast SKUs like glm/glm-4.7-flashx do not). Check the model page at /models for tool-calling support.
• Async streaming hangs: if you're using astream() ensure your outer loop is inside an async def. Sync streaming works from anywhere.