LangChain · Python + JS
LangChain + Leanroute
LangChain's ChatOpenAI and OpenAIEmbeddings accept a base_url override. Point that at Leanroute and every LangChain workflow you have — chains, agents, RAG pipelines, streaming — keeps working, but you can now route across every provider we support instead of just OpenAI.
1. Get a gateway key
From /dashboard/keys create a key labeled "langchain". Copy the gw_live_* value — it's shown only once.
2. Python: chat
pip install langchain-openai if you don't already have it. Then:
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
model="anthropic/claude-sonnet-5", # or openai/gpt-5.6-terra, deepseek/deepseek-v4-flash, etc.
api_key="gw_live_YOUR_KEY",
base_url="https://api.leanroute.dev/v1",
)
print(llm.invoke("Explain retrieval-augmented generation in one sentence.").content)The model string uses our canonical provider/model form. Full list at /docs/models.
Or pip install langchain-leanroute (PyPI) and use ChatLeanroute: the same ChatOpenAI, with the base URL built in, the key read from LEANROUTE_API_KEY, and agent-run options (step 6).
3. Python: streaming
for chunk in llm.stream("Write a haiku about caching."):
print(chunk.content, end="", flush=True)Server-sent events are forwarded from the upstream provider chunk-by-chunk with no intermediate buffering — your first token latency matches going direct.
4. Python: embeddings
from langchain_openai import OpenAIEmbeddings
embed = OpenAIEmbeddings(
model="openai/text-embedding-3-small",
api_key="gw_live_YOUR_KEY",
base_url="https://api.leanroute.dev/v1",
)
vec = embed.embed_query("what is prompt caching?")
print(len(vec)) # 1536Use the same key. Any LangChain vector-store integration (Chroma, Pinecone, pgvector, Weaviate, etc.) accepts an embedding_function argument — pass this embed and your entire indexing + query flow now runs through Leanroute.
5. JavaScript / TypeScript
npm install @langchain/openai. The JS package uses configuration.baseURL (nested) instead of the flat base_url Python uses.
import { ChatOpenAI } from "@langchain/openai";
const llm = new ChatOpenAI({
model: "anthropic/claude-sonnet-5",
apiKey: "gw_live_YOUR_KEY",
configuration: {
baseURL: "https://api.leanroute.dev/v1",
},
});
const res = await llm.invoke("Explain retrieval-augmented generation in one sentence.");
console.log(res.content);Or use the @leanroute/langchain package
ChatLeanroute returns a normal ChatOpenAI with the base URL built in, typed model ids, and an agent option that sets the agent run headers for you. For Python, use langchain-leanroute (step 6).
npm install @leanroute/langchain @langchain/openai @langchain/coreimport { ChatLeanroute, newRunId } from "@leanroute/langchain";
// Reads LEANROUTE_API_KEY; base URL is built in.
const llm = ChatLeanroute({
model: "deepseek/deepseek-v4-1-flash",
agent: { name: "research-agent", sessionId: newRunId() },
});
const res = await llm.invoke("Explain retrieval-augmented generation in one sentence.");
console.log(res.content);6. LangGraph agents and agent runs
LangGraph and LangChain agents work end-to-end because Leanroute forwards the tools and tool_calls fields to the upstream unchanged; any model with tool calling works. Add two headers and each run shows up on Agent runs with its cost, models and every tool the model asked for. pip install langchain-leanroute langgraph for this example. Plain ChatOpenAI works too: pass the same values as X-Agent-Name and X-Agent-Session-Id in default_headers.
from langchain_core.tools import tool
from langgraph.prebuilt import create_react_agent
from langchain_leanroute import ChatLeanroute, new_run_id
@tool
def get_weather(city: str) -> str:
"""Return the current temperature for a city."""
return f"{city}: 22°C"
def run(task: str) -> str:
# One model per run: the session id groups every call in this run
# on the Leanroute Agent runs page (cost, models, tools called).
# Reads LEANROUTE_API_KEY; base URL is built in.
llm = ChatLeanroute(
model="anthropic/claude-sonnet-5",
agent_name="weather-agent",
session_id=new_run_id(),
)
agent = create_react_agent(llm, tools=[get_weather])
result = agent.invoke({"messages": [("user", task)]})
return result["messages"][-1].content
print(run("What's the weather in Singapore?"))Optional: cross-provider fallback
Enable the cross-provider failover toggle in /dashboard/settings and any 5xx from your chosen provider (e.g. Anthropic 529 overload) will retry on the cheapest same-tier alternative. Your LangChain code doesn't change; the retry is transparent from the SDK's perspective. Response header x-gateway-provider tells you which provider actually served the request.
Troubleshooting
- 401 unauthenticated: the key was rotated or revoked. Issue a new one at /dashboard/keys.
- 400 unknown_model: the model name doesn't match our canonical list. The provider prefix is required (use
anthropic/claude-sonnet-5, notclaude-sonnet-5). - Tool calls not firing: confirm the model supports tools upstream (all flagship models do; some cheap_fast SKUs like
glm/glm-4.7-flashxdo not). Check the model page at /models for tool-calling support. - Async streaming hangs: if you're using
astream()ensure your outer loop is inside anasync def. Sync streaming works from anywhere.