LlamaIndex + Leanroute

1. Get a gateway key

From /dashboard/keys create a key labeled "llamaindex". Copy the gw_live_* value.

2. Python: single-call chat

pip install llama-index-llms-openai llama-index-embeddings-openai if you don't already have them.

from llama_index.llms.openai import OpenAI

llm = OpenAI(
    model="anthropic/claude-sonnet-4-6",   # or openai/gpt-4o, deepseek/deepseek-v4-flash, etc.
    api_key="gw_live_YOUR_KEY",
    api_base="https://api.leanroute.dev/v1",
)

print(llm.complete("Explain vector databases in one sentence.").text)

3. Python: set once, use everywhere

LlamaIndex's Settings singleton is the recommended integration pattern. Set both llm and embed_model at app startup, and every downstream construct picks them up implicitly.

from llama_index.core import Settings
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding

# Set once — every query engine, agent, and workflow you build after
# this point will route through Leanroute.
Settings.llm = OpenAI(
    model="anthropic/claude-sonnet-4-6",
    api_key="gw_live_YOUR_KEY",
    api_base="https://api.leanroute.dev/v1",
)
Settings.embed_model = OpenAIEmbedding(
    model="openai/text-embedding-3-small",
    api_key="gw_live_YOUR_KEY",
    api_base="https://api.leanroute.dev/v1",
)

4. Python: RAG pipeline

With Settings configured above, a standard LlamaIndex RAG pipeline works unchanged — the ingest embeds via Leanroute, the query generation completes via Leanroute:

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

# Settings.llm + Settings.embed_model above make this route through
# Leanroute for both the embedding and the answer generation.
documents = SimpleDirectoryReader("./docs").load_data()
index = VectorStoreIndex.from_documents(documents)

query_engine = index.as_query_engine()
response = query_engine.query("What does the docs corpus say about caching?")
print(response)

5. TypeScript

npm install llamaindex @llamaindex/openai. The TS package places the base URL on additionalSessionOptions.baseURL:

import { OpenAI } from "@llamaindex/openai";

const llm = new OpenAI({
  model: "anthropic/claude-sonnet-4-6",
  apiKey: "gw_live_YOUR_KEY",
  additionalSessionOptions: {
    baseURL: "https://api.leanroute.dev/v1",
  },
});

const res = await llm.complete({ prompt: "Explain vector databases in one sentence." });
console.log(res.text);

Optional: cheaper-model swaps for query synthesis

RAG pipelines often send lots of short synthesis calls after retrieval. Set Settings.llm to a cheap_fast tier model (e.g. openai/gpt-5-nano) for synthesis and reserve the flagship model for user-facing chat. Or enable cheaper-model swaps at /dashboard/settings and Leanroute will do the pick automatically per request.

Troubleshooting

• 401 unauthenticated: issue a new key at /dashboard/keys.
• 400 unknown_model: use the canonical provider/model form, e.g. openai/text-embedding-3-small, not bare text-embedding-3-small.
• Embedding dimension mismatch: if you swap embedding models mid-index, your vector store will error on insert. Re-index from scratch or use two separate indexes.
• Settings ignored: confirm you imported from llama_index.core (not deprecated llama_index.core.service_context, which was retired in v0.10+).