Auto failover across providers
When a provider goes down or rate-limits, requests automatically route to backups. Your users never see an error.
Visual
Failover path at a glance
How a request moves through primary, retry, and fallback.
User request
Incoming API call with payload
Health-aware routing
Picks the fastest healthy provider
Primary provider attempt
Success when healthy, else retry
Retry with backoff
On 429/5xx, try backup provider
Fallback succeeds
Healthy provider returns response
Trace + metrics logged
Every hop recorded for debugging
workflow: "smart_summary"
providers: ["openai:gpt-4o", "anthropic:sonnet", "groq:mixtral"]
on_error: "next_available"
max_attempts: 3
backoff_ms: [400, 800, 1600]
trace_id: req_92f0...
Detect degraded providers
Health windows decide eligibility before the first call is made.
Retry with context
Same payload and routing metadata flow through retries with backoff.
See every attempt
Request logs capture provider, latency, tokens, and status for each hop.
Failover speed
~450ms
Median time to recover after a provider 429/500.
Observability
Full trace
Each hop recorded in request logs with tokens & latency.
Control
Per-request
Opt-in or customize retries for specific workflows.
01 · Prepare
Select providers with health scores and prioritize by latency or cost.
02 · Route
Route requests with retry budgets and per-workflow configs.
03 · Recover
Failover to healthy models with exponential backoff.
04 · Observe
Inspect attempts, tokens, and timing in request logs.
When to use
- ● Critical user flows that must not 500.
- ● Routing across OpenAI, Anthropic, and Groq with cost/latency preferences.
- ● Experiments where you want automatic fallbacks without client changes.
What you get
- ● Health-aware routing before calls are made.
- ● Consistent payloads with structured outputs and streaming.
- ● Full transparency via analytics and request logs.
Programmatic access
Call your workflow—failover is configured in the console
POST https://api.modelriver.com/v1/ai Authorization: Bearer mr_live_your_key { "workflow": "customer-support", "messages": [ { "role": "user", "content": "..." } ] } // Response shows failover attempts { "data": { ... }, "meta": { "attempts": [ { "provider": "openai", "status": "error" }, { "provider": "anthropic", "status": "success" } ] } }
Configure primary/fallback providers in your workflow. The API handles retries automatically and reports every attempt in the response.
Ship uptime your users feel
Pair failover with rate limiting, structured outputs, and analytics to keep experiences fast and predictable.