Early access: Hosted tools is turned on per organization. Contact support to have it enabled for yours. Until then Let the AI call my endpoints can't be chosen on a workflow's General tab, and a workflow's tools are ignored.
What hosted tools are
With function calling, the model asks for a function and your code runs it, sends the result back, and repeats until the model answers. That loop lives in your backend.
With hosted tools, ModelRiver runs the loop. You register HTTPS endpoints on a workflow, each with a name, a description and a schema for its arguments. When a request comes in:
- ModelRiver sends the user's message to the model, with your tools on offer.
- If the model asks for a tool, ModelRiver checks the arguments against the tool's schema and calls your endpoint with a signed
POST. For a tool that changes something, it first asks the end user to approve the call. - Your endpoint returns JSON. ModelRiver gives it to the model.
- Steps 2 and 3 repeat until the model answers, or a limit is reached.
- The answer is delivered like any async request: over WebSocket, and to your webhooks.
Your frontend can show what is happening while it waits ("Reading this week's sales…"), and the user can stop the request. See Progress events.
An example
A store's admin page has an assistant. The workflow sales_chat has two tools, get_sales_summary and get_top_products. Your backend sends:
curl -X POST https://api.modelriver.com/v1/ai/async \ -H "Authorization: Bearer $MODELRIVER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "workflow": "sales_chat", "messages": [ {"role": "user", "content": "How did we do this week, and what sold best?"} ], "end_user": { "id": "user_812", "claims": {"store_id": "st_44", "role": "manager"} } }'The model asks for both tools. ModelRiver calls both endpoints, each with the arguments the model chose and the end_user you sent. The model reads the two results and writes: "Sales are up 12% on last week, at $48,200. The desk lamp sold best, with 310 units."
That request took two model rounds and two tool calls.
Other things teams build this way: an HR assistant that reads an employee's leave balance, a project page that answers "what is blocking the release?", a finance view that explains a customer's last three invoices. In each case the model decides which question to ask your backend, and your backend decides what that user is allowed to know.
Who does what
| ModelRiver | Your backend |
|---|---|
| Offers your tools to the model, on every provider, and only to models that can call tools | Implements each tool as an HTTPS endpoint |
| Checks the model's arguments against the tool's schema before any call | Checks the signature on every call |
Signs every call and passes on the end_user you sent | Decides what to return from end_user, never from the arguments |
| Runs calls in parallel, retries a failed read once, enforces timeouts | Returns JSON, quickly |
| Waits for the end user's approval before each call to a tool that changes something, and sends an approved call once | Shows the approval in your app and forwards the end user's decision with your API key |
| Stops at the limits you set | Sets those limits in the console |
| Shows every round and call in Request Logs |
The model never sees end_user, your signing secret or your endpoint URLs.
Which models can run tools
Tool rounds run only on models that can call tools: the ones with the Tools capsule in the console's model pickers. That is every chat model in the catalog except:
- GPT-6 Astra, GPT-6.1 Sol and GPT-6.1 Terra on OpenAI and Azure OpenAI (their tool calling needs the Responses API; ModelRiver uses Chat Completions)
- GPT-6.1 Sol and GPT-6.1 Terra on Amazon Bedrock, for the same reason
- Cohere Command A Vision
Embedding models can't, and a custom endpoint is never refused: ModelRiver passes the tools on, so check that your server supports them.
What happens with a model that can't call tools:
| When | What happens |
|---|---|
| You save or publish | Refused while tools are on and at least one tool is switched on, naming the slot: "claude-x (backup 1) can't call tools. Pick a model that can, or turn tools off." The editor warns about it before you save, and its model pickers mark such models No tools |
| A request runs | Failover skips that slot, the way it skips a slot over its budget, and tries the next one. The Timeline shows the skip as Skipped, with the reason. It is not counted |
| No slot can call tools | The request ends with tool_model_unsupported before any model is called, and counts nothing. A Playground run pinned to one slot only checks that slot. See How a request ends |
A workflow made Live before this check keeps running: requests use only its models that can call tools. The editor shows a warning, and the workflow can't be published again until its models are fixed or tools are turned off.
The end_user field
A request to a workflow with hosted tools must say who it is for:
1"end_user": { "id": "user_812", "claims": { "store_id": "st_44" } }| Field | Rules |
|---|---|
id | Required. Text, up to 256 characters. The signed-in user in your app |
claims | Optional. A flat object of up to 16 entries. Values are text, numbers or true/false. 2 KB at most |
ModelRiver passes end_user to every tool call inside the signed body, so your endpoint can trust it as much as it trusts your own backend. It is not sent to the model and not added to the prompt.
With session memory, the first end_user.id used on a session owns it. A request for the same session_id with a different id is refused with 409 end_user_mismatch. The conversation remembers the user's question and the final answer; tool calls and their results are not replayed in later turns.
Limits
Every request with hosted tools runs inside limits, so a model that keeps asking for tools can't run forever. You set the first three per workflow, on its Tools tab.
| Limit | Default | Range | What it bounds |
|---|---|---|---|
| Rounds | 5 | 2 to 10 | Model calls in one request. The last allowed round runs with tools off, so the model has to answer: 5 rounds means tools can run in at most 4 |
| Tool calls | 16 | 1 to 64 | Calls to your endpoints in one request |
| Total time | 120 s | 10 to 240 s | The whole request: model rounds, failover and tool calls |
| Timeout per tool | 10 s | 1 to 30 s | One call to one endpoint, its retry included. Set on each tool |
| Tools per workflow | 16 | ||
| Calls at once | 4 | Tool calls of one round running at the same time | |
| Result size | 32 KB | One tool's response body | |
| Results per request | 256 KB | All results of one request together | |
| Argument size | 16 KB | One call's arguments | |
| Approval wait | 10 min | 1 to 60 min | How long a call to a tool that changes something waits for the end user. Set on the Tools tab. Not part of the total time |
| Requests waiting for approval | 50 | Per project. See Approvals |
Your plan may set lower caps on rounds and tool calls, and on how many hosted-tools requests a project can have running at once. The lower limit applies.
When a request reaches its round or tool-call limit, the model gets one last round with tools off. If it answers, the request succeeds. If not, the request ends with the error tool_loop_limit. See what the end of a request looks like.
What counts as a request
Each model round counts as one request on your plan. Tool calls are free.
A question that takes three rounds uses three requests. This holds however the request ends: a request stopped by a limit, by the user or by a provider failure still counts the rounds that ran. A request that is refused before it starts (a 400, 409 or 429) counts nothing, and neither does a request that ends with tool_model_unsupported, a model slot skipped because it can't call tools, or a run in the Playground.
The final response says what the request used:
1"meta": { "tool_loop": { "rounds": 2, "tool_calls": 2 } }tool_calls counts calls sent to your endpoints. A call that the end user declined, or that waited past its time, is not counted. The Usage API reports tool_rounds and tool_calls per day and per month.
What hosted tools don't do yet
- Async only. Send requests to
POST /v1/ai/async. The synchronous endpoint and"stream": truereturn400 tools_require_async; the OpenAI-compatible endpoints return400in the OpenAI error format, with the codemodel_not_supported. - The answer arrives whole. Tool progress is live, but the final answer is not streamed token by token.
- No changes without asking. Each tool is marked as one that only reads data or one that changes something. A tool that reads runs as soon as the model asks. A tool that changes something runs only after the end user approves that call, and there is no setting to skip the question. See Approvals.
- Chat workflows. A workflow can't have hosted tools together with a backend pipeline or a structured output. See Pipeline or tools?.
- No tools from the request. A request to a workflow with hosted tools can't send its own
toolsorresponse_format(400 tools_conflict).
A workflow whose tools are switched off works exactly as before, client-run function calling included.
Next steps
- Quickstart: one tool, end to end
- Endpoint contract: what your endpoint receives and returns
- Security: verify signatures, and what to trust
- Pipeline or tools?: choose between the two
- Approvals: tools that change something, approved by the end user
- Progress events: show progress and a Stop button
- Testing: test calls, the Playground and local endpoints