Documentation

Drop-in OpenAI SDK compatibility

Connect using any SDK that speaks the OpenAI Chat Completions protocol. Change two lines of code: base URL and API key: and get routing, failover, and observability for free.

ModelRiver provides an OpenAI-compatible adapter that lets you connect using any SDK that speaks the OpenAI Chat Completions protocol. This means you can use ModelRiver as a drop-in replacement for the OpenAI API, while still getting all ModelRiver features: intelligent routing, failover, structured outputs, cost tracking, and more.

model is a workflow name, not a provider model. Send the name of a workflow from your project, such as model="support_bot". The workflow picks the provider, model, fallbacks and guardrails. A provider model such as gpt-4o or openai/gpt-4o returns a 404 that tells you which workflow to send instead. Coming from another AI gateway? See From other gateways.

Base URL: https://api.modelriver.com/v1

Supported SDKs

SDKLanguageHow to connect
OpenAI SDKPythonSet base_url + api_key
OpenAI SDKNode.jsSet baseURL + apiKey
LangChainPythonChatOpenAI(openai_api_base=…)
LlamaIndexPythonOpenAILike(api_base=…)
Vercel AI SDKJS/TScreateOpenAI({ baseURL: … }), then modelriver.chat(…)
OpenAI Agents SDKPythonset_default_openai_client(…) + set_default_openai_api("chat_completions") (guide)

Any other SDK or HTTP client that targets the OpenAI Chat Completions REST shape will also work. SDKs that default to the newer Responses API need one setting changed; see POST /v1/responses.

Quick start

Python (OpenAI SDK)

PYTHON
1from openai import OpenAI
2 
3client = OpenAI(
4 base_url="https://api.modelriver.com/v1",
5 api_key="mr_live_YOUR_API_KEY"
6)
7 
8response = client.chat.completions.create(
9 model="my_workflow", # ← your ModelRiver workflow name
10 messages=[
11 {"role": "system", "content": "You are a helpful assistant."},
12 {"role": "user", "content": "What is ModelRiver?"}
13 ],
14 temperature=0.7,
15 max_tokens=500
16)
17 
18print(response.choices[0].message.content)

Python (LangChain)

PYTHON
1from langchain_openai import ChatOpenAI
2 
3llm = ChatOpenAI(
4 openai_api_base="https://api.modelriver.com/v1",
5 openai_api_key="mr_live_YOUR_API_KEY",
6 model="my_workflow"
7)
8 
9response = llm.invoke("What is ModelRiver?")
10print(response.content)

Python (LlamaIndex)

PYTHON
1from llama_index.llms.openai_like import OpenAILike
2 
3llm = OpenAILike(
4 api_base="https://api.modelriver.com/v1",
5 api_key="mr_live_YOUR_API_KEY",
6 model="my_workflow",
7 is_chat_model=True,
8)
9 
10response = llm.complete("What is ModelRiver?")
11print(response.text)

Node.js (OpenAI SDK)

JAVASCRIPT
1import OpenAI from "openai";
2 
3const client = new OpenAI({
4 baseURL: "https://api.modelriver.com/v1",
5 apiKey: "mr_live_YOUR_API_KEY",
6});
7 
8const completion = await client.chat.completions.create({
9 model: "my_workflow",
10 messages: [{ role: "user", content: "What is ModelRiver?" }],
11 temperature: 0.7,
12});
13 
14console.log(completion.choices[0].message.content);

Vercel AI SDK (Next.js)

TYPESCRIPT
1import { createOpenAI } from "@ai-sdk/openai";
2import { generateText } from "ai";
3 
4const modelriver = createOpenAI({
5 baseURL: "https://api.modelriver.com/v1",
6 apiKey: "mr_live_YOUR_API_KEY",
7});
8 
9const { text } = await generateText({
10 // .chat() sends Chat Completions requests. Since AI SDK 5,
11 // modelriver("my_workflow") would call the Responses API instead.
12 model: modelriver.chat("my_workflow"),
13 prompt: "What is ModelRiver?",
14});

Endpoints

All paths below are relative to https://api.modelriver.com. Every request needs Authorization: Bearer mr_live_YOUR_API_KEY.

POST /v1/chat/completions

The primary endpoint. Accepts OpenAI chat completion requests and translates them through ModelRiver's workflow engine.

Headers:

Authorization: Bearer mr_live_YOUR_API_KEY
Content-Type: application/json

Request body:

JSON
1{
2 "model": "my_workflow",
3 "messages": [{"role": "user", "content": "Hello"}],
4 "temperature": 0.7,
5 "max_tokens": 500,
6 "top_p": 0.9,
7 "stream": false
8}
FieldRequiredDescription
model✅Workflow name: letters, digits and underscores. Not a provider model
messages✅Array of {role, content} message objects
temperature❌Sampling temperature (0–2), passed to provider
max_tokens❌Maximum tokens in response
top_p❌Nucleus sampling parameter
stream❌true for SSE streaming, default false
tools❌Array of tool definitions for function calling
tool_choice❌How the model should use tools (auto, none, etc.)
n❌Must be 1 (only single completions supported)

Response (non-streaming):

JSON
1{
2 "id": "chatcmpl-abc123",
3 "object": "chat.completion",
4 "created": 1700000000,
5 "model": "my_workflow",
6 "choices": [{
7 "index": 0,
8 "message": {"role": "assistant", "content": "…"},
9 "finish_reason": "stop"
10 }],
11 "usage": {
12 "prompt_tokens": 10,
13 "completion_tokens": 50,
14 "total_tokens": 60,
15 "prompt_tokens_details": {"cached_tokens": 0, "cache_write_tokens": 0},
16 "completion_tokens_details": {"reasoning_tokens": 0}
17 },
18 "system_fingerprint": "mr-v1"
19}

The response model is always the workflow's exact name, even when ModelRiver resolved a near spelling for you (see How it works). The provider and model that answered are in the response headers.

POST /v1/completions and POST /v1/embeddings

Legacy text completions and embeddings work the same way: model is the name of a workflow whose request type is completion or embedding. The response model is the workflow name, not the provider's embedding model.

PYTHON
1embedding = client.embeddings.create(
2 model="my_embedding_workflow",
3 input="The quick brown fox",
4)

A workflow only answers on the endpoint that matches its request type. Sending a chat workflow to /v1/embeddings returns a 400 with param: "model".

GET /v1/models

Lists the workflows you can call through this API, as OpenAI-format model objects. A workflow appears only when it has a Live version, its request type is chat, completion or embedding, and it is not a backend-pipeline (event-driven) workflow. Backend-pipeline workflows run through the async API instead.

JSON
1{
2 "object": "list",
3 "data": [
4 {
5 "id": "my_workflow",
6 "object": "model",
7 "created": 1700000000,
8 "owned_by": "modelriver",
9 "permission": [],
10 "root": "openai/gpt-4o-mini",
11 "modelriver_metadata": {
12 "provider": "openai",
13 "provider_model": "gpt-4o-mini",
14 "test_mode": false,
15 "request_type": "chat",
16 "revision_number": 3
17 }
18 }
19 ]
20}

id is the value you send as model. root is the Live version's primary provider/model, shown for information only: sending it as model returns a 404. If the primary fails, a backup model may answer instead.

GET /v1/models/:id

Returns one model object in the same shape. :id is a workflow name. A name that isn't callable returns the same 404 body as a chat request (see Error handling).

Bash
curl https://api.modelriver.com/v1/models/my_workflow \
-H "Authorization: Bearer $MODELRIVER_API_KEY"

POST /v1/responses (not supported)

ModelRiver supports /v1/chat/completions, /v1/completions, /v1/embeddings and /v1/models. The OpenAI Responses API isn't supported, and POST /v1/responses returns an OpenAI-style 404 that says so. Two SDKs call it by default:

  • Vercel AI SDK 5 and later: use modelriver.chat("my_workflow") instead of modelriver("my_workflow").
  • OpenAI Agents SDK: call set_default_openai_api("chat_completions"). See the OpenAI Agents SDK guide.

GET /v1/provider-models (Legacy)

Returns the original ModelRiver model list grouped by provider, for backward compatibility. These are catalog models, not values you can send as model.

How it works: the model ↔ workflow mapping

The model field in your OpenAI request maps directly to a workflow name in ModelRiver. When you send "model": "my_workflow":

  1. The adapter looks up the workflow named my_workflow in your project
  2. The workflow's Live version picks the provider and model (e.g. openai/gpt-4o-mini) and its backups
  3. All workflow features apply: failover, structured outputs, system instructions, guardrails, budgets
  4. The response is translated back into OpenAI format

This means you can change the underlying provider/model in the ModelRiver console, and publish it, without changing any client code.

Name matching. Workflow names use letters, digits and underscores. If no workflow has the exact name you sent, ModelRiver tries the name with separators normalized: my-workflow resolves to my_workflow (or myworkflow, the name the console stores if you typed my-workflow when creating it). Case is not normalized: My_Workflow does not match my_workflow, and the error suggests the right spelling. If a spelling matches more than one workflow, you get a 404 listing them; send the exact name.

When the model value doesn't match any workflow, the 404 explains why and what to send. It looks like this for a provider model:

JSON
1{
2 "error": {
3 "message": "'gpt-4o' is a provider model. ModelRiver's model field takes a workflow name. Workflow 'support_bot' in this project runs openai/gpt-4o — send model=\"support_bot\".",
4 "type": "invalid_request_error",
5 "param": "model",
6 "code": "model_not_found",
7 "modelriver": {
8 "reason": "provider_model",
9 "suggestions": ["support_bot"],
10 "docs_url": "https://modelriver.com/docs/api/gateway-migration"
11 }
12 }
13}

Every reason and its message are listed in Error handling.

See which model ran

Successful responses carry these headers:

HeaderValue
x-modelriver-workflowThe workflow that handled the request (its exact name)
x-modelriver-revisionThe workflow revision that ran, e.g. 3 for R3
x-modelriver-model-resolved-fromOnly when the name you sent was resolved to a different spelling, e.g. my-workflow
x-modelriver-providerThe provider that answered, e.g. openai
x-modelriver-modelThe provider model that answered, e.g. gpt-4o-mini. Differs from the primary when a backup answered
PYTHON
1raw = client.chat.completions.with_raw_response.create(
2 model="my_workflow",
3 messages=[{"role": "user", "content": "Hello"}],
4)
5print(raw.headers.get("x-modelriver-provider"), raw.headers.get("x-modelriver-model"))
6completion = raw.parse()

Each request also appears in Request Logs with every attempt, including failed ones.

Unsupported parameters

The following OpenAI parameters are not supported and will return an error:

  • functions / function_call: Legacy function calling (use tools instead)
  • logprobs / top_logprobs: Log probabilities
  • n > 1: Only single completions are supported

Observability

All requests through the compatibility layer are:

  • ✅ Logged in Request Logs (with seed_batch prefix compat:openai:)
  • ✅ Tracked with telemetry metrics (modelriver.openai_compat.*)
  • ✅ Subject to rate limiting and usage tracking
  • ✅ Visible in the ModelRiver dashboard

Next steps