Docs

Documentation

ModelRiver Docs

OpenAI SDK Compatibility

ModelRiver provides an OpenAI-compatible adapter that lets you connect using any SDK that speaks the OpenAI Chat Completions protocol. This means you can use ModelRiver as a drop-in replacement for the OpenAI API, while still getting all ModelRiver features: intelligent routing, failover, structured outputs, cost tracking, and more.

Supported SDKs

SDKLanguageHow to connect
OpenAI SDKPythonSet base_url + api_key
OpenAI SDKNode.jsSet baseURL + apiKey
LangChainPythonChatOpenAI(openai_api_base=…)
LlamaIndexPythonOpenAI(api_base=…)
Vercel AIJS/TScreateOpenAI({ baseURL: … })

Any other SDK or HTTP client that targets the OpenAI REST shape will also work.

Quick Start

Python (OpenAI SDK)

PYTHON
1from openai import OpenAI
2 
3client = OpenAI(
4 base_url="https://api.modelriver.com/v1",
5 api_key="mr_live_YOUR_API_KEY"
6)
7 
8response = client.chat.completions.create(
9 model="my_workflow", # ← your ModelRiver workflow name
10 messages=[
11 {"role": "system", "content": "You are a helpful assistant."},
12 {"role": "user", "content": "What is ModelRiver?"}
13 ],
14 temperature=0.7,
15 max_tokens=500
16)
17 
18print(response.choices[0].message.content)

Python (LangChain)

PYTHON
1from langchain_openai import ChatOpenAI
2 
3llm = ChatOpenAI(
4 openai_api_base="https://api.modelriver.com/v1",
5 openai_api_key="mr_live_YOUR_API_KEY",
6 model="my_workflow"
7)
8 
9response = llm.invoke("What is ModelRiver?")
10print(response.content)

Python (LlamaIndex)

PYTHON
1from llama_index.llms.openai import OpenAI
2 
3llm = OpenAI(
4 api_base="https://api.modelriver.com/v1",
5 api_key="mr_live_YOUR_API_KEY",
6 model="my_workflow"
7)
8 
9response = llm.complete("What is ModelRiver?")
10print(response.text)

Node.js (OpenAI SDK)

JAVASCRIPT
1import OpenAI from "openai";
2 
3const client = new OpenAI({
4 baseURL: "https://api.modelriver.com/v1",
5 apiKey: "mr_live_YOUR_API_KEY",
6});
7 
8const completion = await client.chat.completions.create({
9 model: "my_workflow",
10 messages: [{ role: "user", content: "What is ModelRiver?" }],
11 temperature: 0.7,
12});
13 
14console.log(completion.choices[0].message.content);

Vercel AI SDK (Next.js)

TYPESCRIPT
1import { createOpenAI } from "@ai-sdk/openai";
2import { generateText } from "ai";
3 
4const modelriver = createOpenAI({
5 baseURL: "https://api.modelriver.com/v1",
6 apiKey: "mr_live_YOUR_API_KEY",
7});
8 
9const { text } = await generateText({
10 model: modelriver("my_workflow"),
11 prompt: "What is ModelRiver?",
12});

Endpoints

POST /api/v1/chat/completions

The primary endpoint. Accepts OpenAI chat completion requests and translates them through ModelRiver's workflow engine.

Headers:

Authorization: Bearer mr_live_YOUR_API_KEY
Content-Type: application/json

Request body:

JSON
1{
2 "model": "my_workflow",
3 "messages": [{"role": "user", "content": "Hello"}],
4 "temperature": 0.7,
5 "max_tokens": 500,
6 "top_p": 0.9,
7 "stream": false
8}
FieldRequiredDescription
model✅Workflow name (acts as model alias)
messages✅Array of {role, content} message objects
temperature❌Sampling temperature (0–2), passed to provider
max_tokens❌Maximum tokens in response
top_p❌Nucleus sampling parameter
stream❌true for SSE streaming, default false
tools❌Array of tool definitions for function calling
tool_choice❌How the model should use tools (auto, none, etc.)
n❌Must be 1 (only single completions supported)

Response (non-streaming):

JSON
1{
2 "id": "chatcmpl-abc123",
3 "object": "chat.completion",
4 "created": 1700000000,
5 "model": "my_workflow",
6 "choices": [{
7 "index": 0,
8 "message": {"role": "assistant", "content": "…"},
9 "finish_reason": "stop"
10 }],
11 "usage": {
12 "prompt_tokens": 10,
13 "completion_tokens": 50,
14 "total_tokens": 60
15 },
16 "system_fingerprint": "mr-v1"
17}

GET /api/v1/models

Returns your project's workflows as OpenAI-format model objects.

JSON
1{
2 "object": "list",
3 "data": [
4 {
5 "id": "my_workflow",
6 "object": "model",
7 "created": 1700000000,
8 "owned_by": "modelriver",
9 "permission": [],
10 "root": "openai/gpt-4o-mini",
11 "modelriver_metadata": {
12 "provider": "openai",
13 "provider_model": "gpt-4o-mini",
14 "test_mode": false,
15 "request_type": "chat"
16 }
17 }
18 ]
19}

GET /api/v1/provider-models (Legacy)

Returns the original ModelRiver model list grouped by provider, for backward compatibility.

Streaming

Set "stream": true to receive Server-Sent Events (SSE):

PYTHON
1stream = client.chat.completions.create(
2 model="my_workflow",
3 messages=[{"role": "user", "content": "Tell me a story"}],
4 stream=True
5)
6 
7for chunk in stream:
8 if chunk.choices[0].delta.content:
9 print(chunk.choices[0].delta.content, end="")

Each SSE event follows the OpenAI delta format:

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}
 
data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
 
data: [DONE]

Streaming features:

  • Heartbeat comments (: heartbeat) every 15 seconds to keep connections alive
  • 5-minute timeout for long-running requests
  • Proper [DONE] sentinel at stream end

How It Works: The model ↔ Workflow Mapping

The model field in your OpenAI request maps directly to a workflow name in ModelRiver. When you send "model": "my_workflow":

  1. The adapter looks up the workflow named my_workflow in your project
  2. The workflow's configured provider and model (e.g. openai/gpt-4o-mini) is used
  3. All workflow features apply: failover, structured outputs, system instructions, etc.
  4. The response is translated back into OpenAI format

This means you can change the underlying provider/model in the ModelRiver console without changing any client code.

Function Calling (Tool Use)

ModelRiver supports OpenAI-compatible function calling via tools and tool_choice. Tools are passed through to the underlying provider (OpenAI, xAI, Mistral), and tool_calls are returned in standard OpenAI format.

Python Example

PYTHON
1response = client.chat.completions.create(
2 model="my_workflow",
3 messages=[{"role": "user", "content": "What's the weather in Paris?"}],
4 tools=[{
5 "type": "function",
6 "function": {
7 "name": "get_weather",
8 "description": "Get the current weather for a location",
9 "parameters": {
10 "type": "object",
11 "properties": {
12 "location": {"type": "string", "description": "City name"}
13 },
14 "required": ["location"]
15 }
16 }
17 }],
18 tool_choice="auto"
19)
20 
21# Check if the model wants to call a function
22message = response.choices[0].message
23if message.tool_calls:
24 for tool_call in message.tool_calls:
25 print(f"Function: {tool_call.function.name}")
26 print(f"Arguments: {tool_call.function.arguments}")
27else:
28 print(message.content)

Response with tool_calls:

JSON
1{
2 "choices": [{
3 "index": 0,
4 "message": {
5 "role": "assistant",
6 "content": null,
7 "tool_calls": [{
8 "id": "call_abc123",
9 "type": "function",
10 "function": {
11 "name": "get_weather",
12 "arguments": "{\"location\": \"Paris\"}"
13 }
14 }]
15 },
16 "finish_reason": "tool_calls"
17 }]
18}

Note: Function calling support requires the underlying provider/model to support it (e.g., OpenAI GPT-4+, xAI Grok, Mistral). Providers like Google (Gemini) and Anthropic use different tool formats and are not yet supported through this adapter.

Unsupported Parameters

The following OpenAI parameters are not supported and will return an error:

  • functions / function_call: Legacy function calling (use tools instead)
  • logprobs / top_logprobs: Log probabilities
  • n > 1: Only single completions are supported

Error Format

Errors follow the OpenAI error envelope:

JSON
1{
2 "error": {
3 "message": "Workflow 'xyz' does not exist. Create it in the console first.",
4 "type": "invalid_request_error",
5 "param": "model",
6 "code": "model_not_found"
7 }
8}

Common error codes:

HTTPCodeMeaning
400invalid_request_errorMissing/invalid params
401authentication_errorInvalid or missing API key
404model_not_foundWorkflow doesn't exist
400model_not_supportedWorkflow is event-driven (use async API)
502upstream_errorProvider returned an error
503service_unavailableFeature disabled or internal failure

Feature Flag

The OpenAI compatibility layer is controlled by the OPENAI_COMPAT_ENABLED environment variable. Set it to "true" to enable (enabled by default).

When disabled, POST /chat/completions returns:

JSON
1{
2 "error": {
3 "message": "OpenAI compatibility layer is not enabled.",
4 "type": "invalid_request_error",
5 "code": "feature_disabled"
6 }
7}

Observability

All requests through the compatibility layer are:

  • ✅ Logged in Request Logs (with seed_batch prefix compat:openai:)
  • ✅ Tracked with telemetry metrics (modelriver.openai_compat.*)
  • ✅ Subject to rate limiting and usage tracking
  • ✅ Visible in the ModelRiver dashboard

Migration from Direct OpenAI

If you're currently using the OpenAI API directly:

  1. Create a workflow in the ModelRiver console
  2. Change base_url to your ModelRiver API URL
  3. Change api_key to your ModelRiver API key
  4. Change model to your workflow name
  5. That's it: no other code changes needed