LangChain Custom LLM API: Point It at Any Base URL
To use a langchain custom llm api you do not need to write a custom class: the OpenAI-compatible chat wrapper accepts a base URL, a key and a model name. Here is the exact configuration, plus streaming, tools, JSON output and limit handling.
Key points
- ChatOpenAI takes base_url, api_key and model, so no subclassing is needed.
- Use model uncensored and the base URL https://api.llmapistatus.com/v1.
- Tool calling, streaming and JSON mode all pass through the endpoint.
- Cap parallel chains at 8 to stay under the concurrency limit; embeddings are not offered.

The short answer: ChatOpenAI with a different base URL
LangChain talks to OpenAI-style servers through the langchain-openai package. Its ChatOpenAI class lets you override where requests go, which is exactly what a custom endpoint needs. Install the package, then pass three values: the base URL https://api.llmapistatus.com/v1, your key and the model ID uncensored.
Install with pip install -U langchain-openai langchain-core.
import os
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
model="uncensored",
base_url="https://api.llmapistatus.com/v1",
api_key=os.environ["LLM_API_KEY"],
temperature=0.7,
)
print(llm.invoke("Reply with the single word: ready").content)
If that prints a reply, the wiring is correct. You get a key from the API key page; the quickstart shows the same switch for plain SDK users. The endpoint serves POST /v1/chat/completions and GET /v1/models, and those two methods are all the OpenAI wrapper needs for chat.
A word on the term "custom LLM". LangChain also documents a way to write your own chat model by subclassing a base class, which makes sense for APIs that do not speak the OpenAI format. This endpoint does speak it, so subclassing would only add code to maintain. Reach for it only if you need behaviour the wrapper cannot express.
Building a chain with prompts and output parsing
With the model object in place, everything else in LangChain works the way it does with any chat model. Here is a small chain written in the expression-language style: a prompt template, the model, and a string parser.
# reuses llm from the first example: base_url https://api.llmapistatus.com/v1, model uncensored
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
prompt = ChatPromptTemplate.from_messages([
("system", "You are a blunt technical editor. Answer in at most {limit} words."),
("human", "Rewrite this for clarity:\n\n{text}"),
])
chain = prompt | llm | StrOutputParser()
print(chain.invoke({
"limit": 60,
"text": "The thing that we did was basically to make the cache be invalidated on write.",
}))
Nothing in this chain knows or cares that the model sits behind a different domain. That is the benefit of the compatible API: prompts, memory classes, retrievers and agents written against the standard chat interface keep working. What you should check on your side is the prompt itself. A model tuned to answer without content refusals follows instructions literally, so vague system prompts produce vague behaviour. State the role, the format and the length.
One limit is worth knowing before you design a retrieval pipeline. This service is text only. There are no embeddings, so OpenAIEmbeddings pointed at this base URL will fail. Use a local embedding model or another provider for the vector store, and use this endpoint only for generation.
Streaming tokens as they arrive
The endpoint supports server-sent events when you send stream: true, and LangChain turns that into an iterator. You do not set the flag yourself; calling .stream() does it.
# reuses llm from the first example: base_url https://api.llmapistatus.com/v1, model uncensored
for chunk in llm.stream("Write a four-line poem about a failing cron job."):
print(chunk.content, end="", flush=True)
print()
For async servers use astream. The same call exists in the JavaScript library:
import { ChatOpenAI } from "@langchain/openai";
const llm = new ChatOpenAI({
model: "uncensored",
apiKey: process.env.LLM_API_KEY,
configuration: { baseURL: "https://api.llmapistatus.com/v1" },
});
const stream = await llm.stream("Write a four-line poem about a failing cron job.");
for await (const chunk of stream) {
process.stdout.write(chunk.content);
}
Remember that a streamed request keeps its concurrency slot until the last event arrives. If a user closes the browser tab and your server keeps reading the stream, that slot stays taken. Cancel the upstream request when the client disconnects. Usage numbers arrive at the end of the stream, which is where you read the final token counts for billing logs.
Tool calling and agents
Tool calling is available: the endpoint accepts tools and tool_choice, streams tool_calls back, and takes your results as messages with role: "tool". In LangChain you attach tools with bind_tools and let the framework build the messages.
# reuses llm from the first example: base_url https://api.llmapistatus.com/v1, model uncensored
from langchain_core.tools import tool
@tool
def get_order_status(order_id: str) -> str:
'Look up the status of an order by its id.'
return f"Order {order_id} shipped yesterday."
llm_tools = llm.bind_tools([get_order_status])
ai = llm_tools.invoke("Where is order A-1042?")
print(ai.tool_calls)
Running the call and returning the result is the second half of the loop:
# reuses llm from the first example: base_url https://api.llmapistatus.com/v1, model uncensored
# llm_tools comes from the previous snippet
from langchain_core.messages import HumanMessage, ToolMessage
messages = [HumanMessage("Where is order A-1042?")]
ai = llm_tools.invoke(messages)
messages.append(ai)
for call in ai.tool_calls:
result = get_order_status.invoke(call["args"])
messages.append(ToolMessage(content=result, tool_call_id=call["id"]))
print(llm_tools.invoke(messages).content)
Open-weight models vary in how well they pick tools and fill arguments. Keep tool descriptions short and concrete, use few tools per call, and validate arguments in your function instead of trusting them. If an agent loops, lower the temperature and set a maximum number of steps in your agent executor.
Structured and JSON output
JSON mode is supported through response_format: {"type": "json_object"}. In LangChain the easiest route is with_structured_output using the JSON-mode method, which also validates the result against a schema.
# reuses llm from the first example: base_url https://api.llmapistatus.com/v1, model uncensored
from pydantic import BaseModel
class Ticket(BaseModel):
title: str
priority: str
tags: list[str]
structured = llm.with_structured_output(Ticket, method="json_mode")
result = structured.invoke(
"Return JSON with keys title, priority and tags for this report: "
"login page times out on mobile, affects many users."
)
print(result)
JSON mode guarantees that the output is a JSON object, not that it has the fields you wanted. Two habits help. First, name the keys in the prompt, as above; the instruction to produce JSON should be explicit. Second, wrap the call in a retry that reparses on failure. The pydantic check catches missing fields and wrong types, and a single retry with the error message appended usually fixes it.
You can also set response_format directly when you build the model with model_kwargs={"response_format": {"type": "json_object"}}, which is handy for plain chains that end in a JSON parser.
Rate limits, retries and parallel chains
Batch jobs and agents make many calls at once, and this is where a custom endpoint differs from a local model. A full key allows 8 concurrent requests and 300 requests per minute. A trial key allows 2 concurrent and 60 per minute. Past a limit, the API replies with HTTP 429 and the error code rate_limited or concurrency; on temporary overload it replies 429 with Retry-After: 5.
Two LangChain settings handle most of it. The model takes max_retries, which retries transient failures with backoff, and runnables accept a max_concurrency value in their config for batch work:
llm = ChatOpenAI(
model="uncensored",
base_url="https://api.llmapistatus.com/v1",
api_key=os.environ["LLM_API_KEY"],
max_retries=4,
timeout=120,
)
questions = [f"Summarise item {i} in one sentence." for i in range(40)]
answers = llm.batch(questions, config={"max_concurrency": 6})
Setting the cap to 6 on a full key leaves room for two other requests, such as a user-facing call from the same app. On a trial key use 1 or 2. Do not rely on retries alone for a concurrency problem: if the cap is above the limit, retries just recreate the same collision. Reading X-RateLimit-Limit-Requests and X-RateLimit-Limit-Concurrency from the response headers tells you what your key is allowed.
Long inputs are fine up to the 100,000-token window, shared between prompt and completion, and the request body is capped at 8 MB. LangChain text splitters set to a few thousand tokens per chunk keep you far from both.
Troubleshooting a custom endpoint
Most problems with a custom base URL fall into a short list, and each one has a quick test.
- Authentication errors. LangChain also reads the
OPENAI_API_KEYenvironment variable. If you passapi_keyexplicitly, that value wins, but a stale variable can still confuse a second model object you forgot to configure. Pass the key every time, and give the variable another name such asLLM_API_KEY. - Not found errors. The base URL must end in
/v1. LangChain appends/chat/completionsitself, so adding it to the base URL produces a doubled path. - Unknown model. Ask
GET /v1/modelswith the same key and copy the ID it prints. It should beuncensored. - Silent hangs in long chains. Set
timeouton the model and log theX-Request-Idresponse header. If you contact support, that ID lets them find the exact request. - Surprising output. Print the final prompt before it is sent. Most strange answers in LangChain come from a template that filled a variable with something unexpected, not from the model.
When in doubt, reproduce the failing call with plain curl against the same URL. If curl works and LangChain does not, the fault is in your chain; if both fail, check the key and balance.
Checking cost while you build
Chains hide token counts, so measure early. Each response includes usage, and the response headers include X-Balance-USD. The price is $0.15 per 1M input tokens and $0.50 per 1M output tokens, billed from prepaid credit.
A worked example: a retrieval chain that sends 6,000 tokens of context and question and gets 400 tokens back costs 6,000 x $0.15 / 1M = $0.0009 for input plus 400 x $0.50 / 1M = $0.0002 for output, so $0.0011 per call. Running 5,000 such calls in a test sweep costs $5.50. That is useful to know before you leave an agent loop running overnight.
Failed or refused requests are not charged. The pricing page lists top-up amounts from $20 to $500 with bonuses, and new accounts can claim a $0.50 trial credit with one click, enough for a few hundred small test calls while you wire things up.
More guides
Questions and answers
Do I have to subclass LangChain's chat model base class?
No. The endpoint is OpenAI-compatible, so ChatOpenAI with a custom base_url, api_key and model works. Subclassing is only needed for APIs that use a different format.
Can I use OpenAIEmbeddings with this endpoint?
No. The service is text only and has no embeddings. Use a local embedding model or another provider for vectors, and use this endpoint for generation.
Which model name do I pass?
Pass uncensored as the model. GET /v1/models returns the same ID.
Why do I get 429 errors in a batch job?
You exceeded either requests per minute or concurrent requests. Lower max_concurrency below 8 on a full key, or 2 on a trial key, and keep retries enabled.
Does LangChain's streaming work here?
Yes. Calling stream or astream uses server-sent events, and usage is sent at the end of the stream.
Your key is one form away
Create an account, copy the key, change the base URL. That is the whole setup.