Most providers can cache the start of a prompt (system prompt, tools, documents, earlier turns) and charge much less for it on later requests. ModelRiver passes your caching hints to the provider, reads the cached token counts it reports, and prices each request with the model's cache prices.
This is the provider's own cache. It is separate from ModelRiver's workflow cache, which returns a stored response without calling the provider at all.
How each provider caches
| Provider | How caching works | What ModelRiver does with cache_control |
|---|---|---|
| Anthropic | You mark what to cache | Sends it on system, message, tool result and tool blocks. ttl is "5m" (default) or "1h" |
| Amazon Bedrock (Claude, Nova) | You mark what to cache | Adds a cache point after each marked block. Claude: system prompt, messages, tool results and tools, with "ttl": "1h" passed on. Nova: system prompt and messages only, five minutes only. Other Bedrock models: markers removed |
| Qwen | Automatic, or marked | Sends markers on message content as they are; drops markers on tools, which Qwen ignores |
| Custom (OpenAI-compatible) | Depends on the server | Sends markers on message content as they are; drops markers on tools |
| OpenAI, Azure OpenAI, Google Gemini, xAI, Mistral, DeepSeek, Groq, Kimi, MiniMax, Cohere | Automatic | Removes the markers, which these APIs reject or ignore. The provider decides what to cache |
Providers cache only prompts above a minimum length, usually 1,024 tokens or more, and only an exact repeat of the start of the prompt. Put the parts that stay the same first and the parts that change last.
Marking blocks to cache
Add cache_control to a content part. The cache covers everything up to and including that part. Content must be a list of parts to carry a marker, not a plain string.
curl https://api.modelriver.com/v1/chat/completions \ -H "Authorization: Bearer $MODELRIVER_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "support-agent", "messages": [ { "role": "system", "content": [ { "type": "text", "text": "You are a support agent for Acme. <the full 20-page product manual>", "cache_control": {"type": "ephemeral"} } ] }, {"role": "user", "content": "How do I reset my router?"} ] }'You can mark:
- Message content parts in
system,user,assistantandtoolmessages - Tools, with
cache_controlon the tool or on itsfunction. The marker caches all tool definitions up to that one
Add "ttl": "1h" to keep an Anthropic or Bedrock cache entry for an hour instead of five minutes. One-hour writes cost more (see What you pay), so use them for prompts reused less often than every five minutes.
In an agent loop, mark the last part of the latest message (often the latest tool result) on each request. Each request then reads the conversation so far from the cache and writes only the new turn.
Markers stay in the request when failover moves it to a backup, so a backup on another provider applies or drops them as the table above shows.
Cached tokens in the response
prompt_tokens is always all of the input, cached tokens included, and completion_tokens includes reasoning tokens, on every provider. (Anthropic and Bedrock report input without the cache; ModelRiver adds it back so the numbers mean the same thing everywhere.)
On /v1/chat/completions, the usage object (also in the last chunk of a stream) shows the details in OpenAI's format. Streams from a Custom (OpenAI-compatible) server report usage only if the server sends it without being asked:
1"usage": {2 "prompt_tokens": 5050,3 "completion_tokens": 200,4 "total_tokens": 5250,5 "prompt_tokens_details": { "cached_tokens": 4000, "cache_write_tokens": 1000 },6 "completion_tokens_details": { "reasoning_tokens": 0 }7}On /v1/ai, meta.usage has the same counts as cache_read_tokens, cache_write_tokens and reasoning_tokens, plus cache_write_1h_tokens: how many of the cache writes were kept for an hour (Anthropic, Bedrock).
| Field | Meaning |
|---|---|
cached_tokens / cache_read_tokens | Input read from the provider's cache |
cache_write_tokens | Input written to the cache on this request |
reasoning_tokens | Output the model spent thinking (reasoning), when the provider reports it separately. Bedrock doesn't; its reasoning is still counted in completion_tokens |
The same counts appear on each request log in the console and in the monthly Usage API.
What you pay
ModelRiver estimates each request's cost with the model's prices:
- Cache reads at the model's cache read price, often a tenth to a half of the input price
- Cache writes at the model's cache write price: 1.25× input on Anthropic, Bedrock Claude, Qwen and OpenAI GPT-5.6 and later, and the input price on providers without a write charge
- One-hour cache writes (Anthropic, Bedrock) at twice the input price
- Everything else at the input and output prices
A model without a published cache price is priced at its input price. For cache reads that makes the estimate high; for providers that charge extra for writes it makes it low. See Cost analytics for the full formula.