Automatic failover
When a provider fails,a backup model answers.
Give each workflow up to two backup models, from the same provider or a different one. If a call fails for a reason another model can fix, ModelRiver tries the next one and returns the first answer that works.
How failover works
You choose the models once, on the workflow. Your code keeps sending the same request.
- 01
Pick your models
Set a primary model and up to two backups. Mix providers, for example OpenAI first and Anthropic as the backup.
- 02
Call the primary
Each request goes to the primary model first, using your own key for that provider.
- 03
Move on when it fails
Timeouts, dropped connections, rate limits, server errors, and 404 or 408 responses send the request to the next backup.
- 04
See every attempt
The response says which model answered and lists every model it tried. The request log shows the same attempts in order.
The numbers
- 2Backup models you can add to each workflow
- 3Most models tried for one request: the primary and two backups
- 429 · 5xx · 408 · 404Provider errors that move a request to the next backup, along with timeouts and dropped connections
- 1 requestA request that fails over still counts once on your plan
What it looks like
Nothing changes in your request. The response tells you what happened.
// The primary failed with a 503, so the first backup answered
{
"data": { … },
"meta": {
"requested_provider": "openai",
"requested_model": "gpt-4.1",
"used_provider": "anthropic",
"used_model": "claude-haiku-5-5",
"attempts": [
{ "provider": "openai", "model": "gpt-4.1", "status": "error",
"reason": { "message": "Service unavailable" } },
{ "provider": "anthropic", "model": "claude-haiku-5-5", "status": "success" }
]
}
}Backups on another provider
If a provider rejects your key, a backup on the same provider would fail the same way, so ModelRiver skips ahead to a backup on a different provider.
Other reasons to switch
A backup is also tried when a model is unavailable, when a model is over its spend limit, or when a structured output comes back empty or doesn’t match your schema.
Every endpoint
Sync, async and streaming requests and the OpenAI-compatible endpoint all use the same primary and backups.
Good to know
Not every error fails over
Problems with the request itself, like a bad parameter, come back right away. Only failures that another model could fix move on.
No repeat calls to the same model
After a provider error, ModelRiver moves to the next backup instead of calling the same model again. Dropped connections are retried twice, 250 ms apart.
When every model fails
If the primary and both backups fail, you get the error, with every attempt listed so you can see what each provider said.
Connect each provider first
Every attempt uses your key for that provider. Connect a provider in your project before you add one of its models as a backup.
Frequently asked questions
Which errors trigger failover?
Timeouts, dropped connections, rate limits (429), server errors (5xx), 408 and 404. A rejected key fails over only to a backup on a different provider. Other request errors are returned without trying a backup.
How many backups can I add?
Two per workflow. With the primary, that is up to three models for one request.
Can the backup be on a different provider?
Yes. Each backup can use any connected provider and model, so you can put OpenAI, Anthropic, Google or another provider behind each other.
Does a request that fails over count twice?
No. On your ModelRiver plan it counts as one request, however many models were tried.
How do I know which model answered?
The response meta shows the requested and the used provider and model, plus a list of attempts. The request log shows the same attempts in order.
Does failover work with streaming?
Yes. Streaming requests use the same primary and backups as other requests.
Works well with
Add a backup model to a workflow
Start on the free plan, connect two providers, and let the backup answer when the first one fails.