Docs

Documentation

Drop-in OpenAI SDK compatibility

Connect using any SDK that speaks the OpenAI Chat Completions protocol. Change two lines of code: base URL and API key: and get routing, failover, and observability for free.

ModelRiver provides an OpenAI-compatible adapter that lets you connect using any SDK that speaks the OpenAI Chat Completions protocol. This means you can use ModelRiver as a drop-in replacement for the OpenAI API, while still getting all ModelRiver features: intelligent routing, failover, structured outputs, cost tracking, and more.

Supported SDKs

SDKLanguageHow to connect
OpenAI SDKPythonSet base_url + api_key
OpenAI SDKNode.jsSet baseURL + apiKey
LangChainPythonChatOpenAI(openai_api_base=…)
LlamaIndexPythonOpenAI(api_base=…)
Vercel AIJS/TScreateOpenAI({ baseURL: … })

Any other SDK or HTTP client that targets the OpenAI REST shape will also work.

Quick start

Python (OpenAI SDK)

PYTHON
1from openai import OpenAI
2 
3client = OpenAI(
4 base_url="https://api.modelriver.com/v1",
5 api_key="mr_live_YOUR_API_KEY"
6)
7 
8response = client.chat.completions.create(
9 model="my_workflow", # ← your ModelRiver workflow name
10 messages=[
11 {"role": "system", "content": "You are a helpful assistant."},
12 {"role": "user", "content": "What is ModelRiver?"}
13 ],
14 temperature=0.7,
15 max_tokens=500
16)
17 
18print(response.choices[0].message.content)

Python (LangChain)

PYTHON
1from langchain_openai import ChatOpenAI
2 
3llm = ChatOpenAI(
4 openai_api_base="https://api.modelriver.com/v1",
5 openai_api_key="mr_live_YOUR_API_KEY",
6 model="my_workflow"
7)
8 
9response = llm.invoke("What is ModelRiver?")
10print(response.content)

Python (LlamaIndex)

PYTHON
1from llama_index.llms.openai import OpenAI
2 
3llm = OpenAI(
4 api_base="https://api.modelriver.com/v1",
5 api_key="mr_live_YOUR_API_KEY",
6 model="my_workflow"
7)
8 
9response = llm.complete("What is ModelRiver?")
10print(response.text)

Node.js (OpenAI SDK)

JAVASCRIPT
1import OpenAI from "openai";
2 
3const client = new OpenAI({
4 baseURL: "https://api.modelriver.com/v1",
5 apiKey: "mr_live_YOUR_API_KEY",
6});
7 
8const completion = await client.chat.completions.create({
9 model: "my_workflow",
10 messages: [{ role: "user", content: "What is ModelRiver?" }],
11 temperature: 0.7,
12});
13 
14console.log(completion.choices[0].message.content);

Vercel AI SDK (Next.js)

TYPESCRIPT
1import { createOpenAI } from "@ai-sdk/openai";
2import { generateText } from "ai";
3 
4const modelriver = createOpenAI({
5 baseURL: "https://api.modelriver.com/v1",
6 apiKey: "mr_live_YOUR_API_KEY",
7});
8 
9const { text } = await generateText({
10 model: modelriver("my_workflow"),
11 prompt: "What is ModelRiver?",
12});

Endpoints

POST /api/v1/chat/completions

The primary endpoint. Accepts OpenAI chat completion requests and translates them through ModelRiver's workflow engine.

Headers:

Authorization: Bearer mr_live_YOUR_API_KEY
Content-Type: application/json

Request body:

JSON
1{
2 "model": "my_workflow",
3 "messages": [{"role": "user", "content": "Hello"}],
4 "temperature": 0.7,
5 "max_tokens": 500,
6 "top_p": 0.9,
7 "stream": false
8}
FieldRequiredDescription
model✅Workflow name (acts as model alias)
messages✅Array of {role, content} message objects
temperature❌Sampling temperature (0–2), passed to provider
max_tokens❌Maximum tokens in response
top_p❌Nucleus sampling parameter
stream❌true for SSE streaming, default false
tools❌Array of tool definitions for function calling
tool_choice❌How the model should use tools (auto, none, etc.)
n❌Must be 1 (only single completions supported)

Response (non-streaming):

JSON
1{
2 "id": "chatcmpl-abc123",
3 "object": "chat.completion",
4 "created": 1700000000,
5 "model": "my_workflow",
6 "choices": [{
7 "index": 0,
8 "message": {"role": "assistant", "content": "…"},
9 "finish_reason": "stop"
10 }],
11 "usage": {
12 "prompt_tokens": 10,
13 "completion_tokens": 50,
14 "total_tokens": 60
15 },
16 "system_fingerprint": "mr-v1"
17}

GET /api/v1/models

Returns your project's workflows as OpenAI-format model objects.

JSON
1{
2 "object": "list",
3 "data": [
4 {
5 "id": "my_workflow",
6 "object": "model",
7 "created": 1700000000,
8 "owned_by": "modelriver",
9 "permission": [],
10 "root": "openai/gpt-4o-mini",
11 "modelriver_metadata": {
12 "provider": "openai",
13 "provider_model": "gpt-4o-mini",
14 "test_mode": false,
15 "request_type": "chat"
16 }
17 }
18 ]
19}

GET /api/v1/provider-models (Legacy)

Returns the original ModelRiver model list grouped by provider, for backward compatibility.

How it works: the model ↔ workflow mapping

The model field in your OpenAI request maps directly to a workflow name in ModelRiver. When you send "model": "my_workflow":

  1. The adapter looks up the workflow named my_workflow in your project
  2. The workflow's configured provider and model (e.g. openai/gpt-4o-mini) is used
  3. All workflow features apply: failover, structured outputs, system instructions, etc.
  4. The response is translated back into OpenAI format

This means you can change the underlying provider/model in the ModelRiver console without changing any client code.

Unsupported parameters

The following OpenAI parameters are not supported and will return an error:

  • functions / function_call: Legacy function calling (use tools instead)
  • logprobs / top_logprobs: Log probabilities
  • n > 1: Only single completions are supported

Observability

All requests through the compatibility layer are:

  • ✅ Logged in Request Logs (with seed_batch prefix compat:openai:)
  • ✅ Tracked with telemetry metrics (modelriver.openai_compat.*)
  • ✅ Subject to rate limiting and usage tracking
  • ✅ Visible in the ModelRiver dashboard

Feature flag

The OpenAI compatibility layer is controlled by the OPENAI_COMPAT_ENABLED environment variable. Set it to "true" to enable (enabled by default).

When disabled, POST /chat/completions returns:

JSON
1{
2 "error": {
3 "message": "OpenAI compatibility layer is not enabled.",
4 "type": "invalid_request_error",
5 "code": "feature_disabled"
6 }
7}

Next steps