Function Calling & Tool Use

ModelRiver supports OpenAI-compatible function calling via tools, tool_choice and parallel_tool_calls, on every provider. You always send tools, tool calls and tool results in the OpenAI format; ModelRiver translates them for providers with their own format (Anthropic, Google Gemini, Amazon Bedrock, Cohere) and returns tool calls in the OpenAI format. Because the format is the same everywhere, a conversation with tool calls can fail over from one provider to another.

How function calling works

Define tools: Describe the functions your application can execute
Send the request: Include tool definitions alongside your messages
Receive tool_calls: The model returns structured function calls instead of text
Execute functions: Your application runs the requested functions
Return results: Send function results back to continue the conversation

Basic example

Python

PYTHON

1from openai import OpenAI
2 
3client = OpenAI(
4    base_url="https://api.modelriver.com/v1",
5    api_key="mr_live_YOUR_API_KEY"
6)
7 
8response = client.chat.completions.create(
9    model="my_workflow",
10    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
11    tools=[{
12        "type": "function",
13        "function": {
14            "name": "get_weather",
15            "description": "Get the current weather for a location",
16            "parameters": {
17                "type": "object",
18                "properties": {
19                    "location": {
20                        "type": "string",
21                        "description": "City name"
22                    },
23                    "unit": {
24                        "type": "string",
25                        "enum": ["celsius", "fahrenheit"],
26                        "description": "Temperature unit"
27                    }
28                },
29                "required": ["location"]
30            }
31        }
32    }],
33    tool_choice="auto"
34)
35 
36# Check if the model wants to call a function
37message = response.choices[0].message
38if message.tool_calls:
39    for tool_call in message.tool_calls:
40        print(f"Function: {tool_call.function.name}")
41        print(f"Arguments: {tool_call.function.arguments}")
42else:
43    print(message.content)

Node.js

JAVASCRIPT

1import OpenAI from "openai";
2 
3const client = new OpenAI({
4  baseURL: "https://api.modelriver.com/v1",
5  apiKey: "mr_live_YOUR_API_KEY",
6});
7 
8const response = await client.chat.completions.create({
9  model: "my_workflow",
10  messages: [{ role: "user", content: "What's the weather in Paris?" }],
11  tools: [{
12    type: "function",
13    function: {
14      name: "get_weather",
15      description: "Get the current weather for a location",
16      parameters: {
17        type: "object",
18        properties: {
19          location: { type: "string", description: "City name" },
20          unit: { type: "string", enum: ["celsius", "fahrenheit"] },
21        },
22        required: ["location"],
23      },
24    },
25  }],
26  tool_choice: "auto",
27});
28 
29const message = response.choices[0].message;
30if (message.tool_calls) {
31  for (const toolCall of message.tool_calls) {
32    console.log(`Function: ${toolCall.function.name}`);
33    console.log(`Arguments: ${toolCall.function.arguments}`);
34  }
35} else {
36  console.log(message.content);
37}

Response with tool_calls

When the model decides to call a function, the response includes a tool_calls array:

JSON

1{
2  "choices": [{
3    "index": 0,
4    "message": {
5      "role": "assistant",
6      "content": null,
7      "tool_calls": [{
8        "id": "call_abc123",
9        "type": "function",
10        "function": {
11          "name": "get_weather",
12          "arguments": "{\"location\": \"Paris\", \"unit\": \"celsius\"}"
13        }
14      }]
15    },
16    "finish_reason": "tool_calls"
17  }]
18}

Response fields

Field	Description
`tool_calls[].id`	Unique identifier for this tool call
`tool_calls[].type`	Always `"function"`
`tool_calls[].function.name`	Name of the function to execute
`tool_calls[].function.arguments`	JSON string of function arguments
`tool_calls[].extra_content`	Present only when the provider needs data sent back with the call, such as a Gemini thought signature. Send it back unchanged
`finish_reason`	Set to `"tool_calls"` when the model wants to call functions
`content`	Typically `null` when tool_calls are present

Multi-turn tool conversations

After executing the requested function, send the result back to continue the conversation:

PYTHON

1import json
2 
3# Step 1: Initial request with tools
4messages = [{"role": "user", "content": "What's the weather in Paris?"}]
5 
6response = client.chat.completions.create(
7    model="my_workflow",
8    messages=messages,
9    tools=tools
10)
11 
12message = response.choices[0].message
13 
14# Step 2: Execute the function
15if message.tool_calls:
16    # Add the assistant's message exactly as received (with tool_calls and
17    # any extra_content on them)
18    messages.append(message.model_dump(exclude_none=True))
19    
20    for tool_call in message.tool_calls:
21        # Execute your function
22        if tool_call.function.name == "get_weather":
23            args = json.loads(tool_call.function.arguments)
24            result = get_weather(args["location"])  # Your function
25            
26            # Add the function result
27            messages.append({
28                "role": "tool",
29                "tool_call_id": tool_call.id,
30                "content": json.dumps(result)
31            })
32    
33    # Step 3: Get the final response
34    final_response = client.chat.completions.create(
35        model="my_workflow",
36        messages=messages,
37        tools=tools
38    )
39    
40    print(final_response.choices[0].message.content)
41    # Output: "The weather in Paris is currently 18°C and partly cloudy."

Each tool result is a tool message with the tool_call_id of the call it answers. Its content is usually a string; a list of text parts also works. When the model made several calls at once, send one tool message per call, in the order of the calls, right after the assistant message.

Streaming tool calls

With stream: true, tool calls arrive as deltas in choices[0].delta.tool_calls. The first delta for a call has its index, id and function.name; later deltas for the same index add pieces of function.arguments. The last chunk has finish_reason: "tool_calls".

PYTHON

1stream = client.chat.completions.create(
2    model="my_workflow",
3    messages=messages,
4    tools=tools,
5    stream=True,
6)
7 
8calls = {}
9for chunk in stream:
10    if not chunk.choices:
11        continue
12    for delta in chunk.choices[0].delta.tool_calls or []:
13        call = calls.setdefault(delta.index, {"id": None, "name": "", "arguments": ""})
14        if delta.id:
15            call["id"] = delta.id
16        if delta.function and delta.function.name:
17            call["name"] = delta.function.name
18        if delta.function and delta.function.arguments:
19            call["arguments"] += delta.function.arguments

ModelRiver doesn't stream from Amazon Bedrock yet, so a Bedrock workflow sends the whole reply, tool calls included, in one go.

Tool choice options

Control how the model uses tools with the tool_choice parameter:

Value	Behaviour
`"auto"`	Model decides whether to call a function (default)
`"none"`	Model will not call any functions
`"required"`	Model must call at least one function
`{"type": "function", "function": {"name": "get_weather"}}`	Force a specific function

Set parallel_tool_calls: false to ask for at most one tool call per reply.

PYTHON

1# Force the model to call a specific function
2response = client.chat.completions.create(
3    model="my_workflow",
4    messages=messages,
5    tools=tools,
6    tool_choice={
7        "type": "function",
8        "function": {"name": "get_weather"}
9    }
10)

Multiple tools

Define multiple tools and let the model choose which to call:

PYTHON

1tools = [
2    {
3        "type": "function",
4        "function": {
5            "name": "get_weather",
6            "description": "Get weather for a location",
7            "parameters": {
8                "type": "object",
9                "properties": {
10                    "location": {"type": "string"}
11                },
12                "required": ["location"]
13            }
14        }
15    },
16    {
17        "type": "function",
18        "function": {
19            "name": "search_restaurants",
20            "description": "Search for restaurants near a location",
21            "parameters": {
22                "type": "object",
23                "properties": {
24                    "location": {"type": "string"},
25                    "cuisine": {"type": "string"},
26                    "max_results": {"type": "integer", "default": 5}
27                },
28                "required": ["location"]
29            }
30        }
31    },
32    {
33        "type": "function",
34        "function": {
35            "name": "book_reservation",
36            "description": "Book a table at a restaurant",
37            "parameters": {
38                "type": "object",
39                "properties": {
40                    "restaurant_id": {"type": "string"},
41                    "date": {"type": "string", "format": "date"},
42                    "party_size": {"type": "integer"},
43                    "time": {"type": "string"}
44                },
45                "required": ["restaurant_id", "date", "party_size"]
46            }
47        }
48    }
49]

The model may call multiple tools in a single response. Handle each tool call individually and return all results.

Native API (/v1/ai)

POST /v1/ai accepts the same tools, tool_choice and parallel_tool_calls fields and the same tool messages. Without streaming, data holds the provider response in the OpenAI format, so tool calls are in data.choices[0].message.tool_calls. With stream: true it streams the same chunks as above.

Provider support

Tool calling works on every provider. The workflow's model must support tool use (models marked Tools on the providers page); otherwise the provider returns an error.

Provider	How tools are sent	Notes
OpenAI, Azure OpenAI, xAI, DeepSeek, Qwen, Groq, Kimi, MiniMax, Custom	Passed through in the OpenAI format
Mistral	Passed through in the OpenAI format	Mistral only accepts 9-character tool call ids, so ids from another provider are mapped to such ids
Anthropic	Translated to `tool_use` and `tool_result` blocks	All `tool_choice` values and `parallel_tool_calls`
Google Gemini	Translated to `functionDeclarations`, `functionCall` and `functionResponse`	`parallel_tool_calls` is ignored. `additionalProperties` is removed from tool schemas. Gemini 2.x models can't combine tools with structured output, so the schema is dropped when tools are sent. Gemini 3 models return a thought signature in `extra_content`; send the assistant message back unchanged
Amazon Bedrock	Translated to Converse `toolConfig`, `toolUse` and `toolResult`	`tool_choice: "none"` leaves the tools out; once the conversation has tool calls, Bedrock needs the tools, so the model may still call one. `parallel_tool_calls` is ignored. Replies are not streamed
Cohere	Passed in Cohere's tools format	A named `tool_choice` sends only that tool and requires it. `parallel_tool_calls` is ignored

Best practices

Write clear descriptions: Function descriptions directly affect the model's decision to call them
Use required fields: Specify which parameters are required to avoid incomplete calls
Validate arguments: Always validate the parsed arguments before executing functions
Handle missing calls: The model may respond with text instead of tool calls even when tools are available
Limit tool count: Too many tools can confuse the model; keep the list focused
Use enum for constrained values: Reduces errors in function arguments
Set timeouts for function execution: Don't let tool execution block indefinitely

Next steps

Streaming: Stream responses token by token
Request types: Different AI operation types
Error handling: Handle function calling errors

How function calling works

Basic example

Python

Node.js

Response with tool_calls

Response fields

Multi-turn tool conversations

Streaming tool calls

Tool choice options

Multiple tools

Native API (/v1/ai)

Provider support

Best practices

Next steps