LangChain · Python + JS

LangChain + Leanroute

LangChain's ChatOpenAI and OpenAIEmbeddings accept a base_url override. Point that at Leanroute and every LangChain workflow you have — chains, agents, RAG pipelines, streaming — keeps working, but you can now route across every provider we support instead of just OpenAI.

1. Get a gateway key

From /dashboard/keys create a key labeled "langchain". Copy the gw_live_* value — it's shown only once.

2. Python: chat

pip install langchain-openai if you don't already have it. Then:

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="anthropic/claude-sonnet-5",   # or openai/gpt-5.6-terra, deepseek/deepseek-v4-flash, etc.
    api_key="gw_live_YOUR_KEY",
    base_url="https://api.leanroute.dev/v1",
)

print(llm.invoke("Explain retrieval-augmented generation in one sentence.").content)

The model string uses our canonical provider/model form. Full list at /docs/models.

Or pip install langchain-leanroute (PyPI) and use ChatLeanroute: the same ChatOpenAI, with the base URL built in, the key read from LEANROUTE_API_KEY, and agent-run options (step 6).

3. Python: streaming

for chunk in llm.stream("Write a haiku about caching."):
    print(chunk.content, end="", flush=True)

Server-sent events are forwarded from the upstream provider chunk-by-chunk with no intermediate buffering — your first token latency matches going direct.

4. Python: embeddings

from langchain_openai import OpenAIEmbeddings

embed = OpenAIEmbeddings(
    model="openai/text-embedding-3-small",
    api_key="gw_live_YOUR_KEY",
    base_url="https://api.leanroute.dev/v1",
)

vec = embed.embed_query("what is prompt caching?")
print(len(vec))  # 1536

Use the same key. Any LangChain vector-store integration (Chroma, Pinecone, pgvector, Weaviate, etc.) accepts an embedding_function argument — pass this embed and your entire indexing + query flow now runs through Leanroute.

5. JavaScript / TypeScript

npm install @langchain/openai. The JS package uses configuration.baseURL (nested) instead of the flat base_url Python uses.

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
  model: "anthropic/claude-sonnet-5",
  apiKey: "gw_live_YOUR_KEY",
  configuration: {
    baseURL: "https://api.leanroute.dev/v1",
  },
});

const res = await llm.invoke("Explain retrieval-augmented generation in one sentence.");
console.log(res.content);

Or use the @leanroute/langchain package

ChatLeanroute returns a normal ChatOpenAI with the base URL built in, typed model ids, and an agent option that sets the agent run headers for you. For Python, use langchain-leanroute (step 6).

npm install @leanroute/langchain @langchain/openai @langchain/core
import { ChatLeanroute, newRunId } from "@leanroute/langchain";

// Reads LEANROUTE_API_KEY; base URL is built in.
const llm = ChatLeanroute({
  model: "deepseek/deepseek-v4-1-flash",
  agent: { name: "research-agent", sessionId: newRunId() },
});

const res = await llm.invoke("Explain retrieval-augmented generation in one sentence.");
console.log(res.content);

6. LangGraph agents and agent runs

LangGraph and LangChain agents work end-to-end because Leanroute forwards the tools and tool_calls fields to the upstream unchanged; any model with tool calling works. Add two headers and each run shows up on Agent runs with its cost, models and every tool the model asked for. pip install langchain-leanroute langgraph for this example. Plain ChatOpenAI works too: pass the same values as X-Agent-Name and X-Agent-Session-Id in default_headers.

from langchain_core.tools import tool
from langgraph.prebuilt import create_react_agent
from langchain_leanroute import ChatLeanroute, new_run_id

@tool
def get_weather(city: str) -> str:
    """Return the current temperature for a city."""
    return f"{city}: 22°C"

def run(task: str) -> str:
    # One model per run: the session id groups every call in this run
    # on the Leanroute Agent runs page (cost, models, tools called).
    # Reads LEANROUTE_API_KEY; base URL is built in.
    llm = ChatLeanroute(
        model="anthropic/claude-sonnet-5",
        agent_name="weather-agent",
        session_id=new_run_id(),
    )
    agent = create_react_agent(llm, tools=[get_weather])
    result = agent.invoke({"messages": [("user", task)]})
    return result["messages"][-1].content

print(run("What's the weather in Singapore?"))

Optional: cross-provider fallback

Enable the cross-provider failover toggle in /dashboard/settings and any 5xx from your chosen provider (e.g. Anthropic 529 overload) will retry on the cheapest same-tier alternative. Your LangChain code doesn't change; the retry is transparent from the SDK's perspective. Response header x-gateway-provider tells you which provider actually served the request.

Troubleshooting

  • 401 unauthenticated: the key was rotated or revoked. Issue a new one at /dashboard/keys.
  • 400 unknown_model: the model name doesn't match our canonical list. The provider prefix is required (use anthropic/claude-sonnet-5, not claude-sonnet-5).
  • Tool calls not firing: confirm the model supports tools upstream (all flagship models do; some cheap_fast SKUs like glm/glm-4.7-flashx do not). Check the model page at /models for tool-calling support.
  • Async streaming hangs: if you're using astream() ensure your outer loop is inside an async def. Sync streaming works from anywhere.