LlamaIndex alternative
LlamaIndex builds the retrieval.ModelRiver runs the model calls.
Keep your indexes, retrievers and query engines. Route the model calls through ModelRiver to add backups, request logs, structured output and limits, without rewriting your pipeline.
Where RAG pipelines break in production
Retrieval is rarely what fails. The trouble is usually the model call at the end of the pipeline.
A provider goes down
When the model behind your query engine returns errors or rate limits, the query fails unless you wrote fallback code.
Hard to see what happened
Tracing comes from instrumentation you connect to a separate tool.
Answers in the wrong shape
Parsing the output is up to your code, and models don’t always follow the format.
Costs that grow quietly
Every query sends retrieved context to the model. Without caps, a busy index can run up the bill.
Keep the retrieval, add the production layer
ModelRiver speaks the OpenAI API. Point your LlamaIndex LLM at it and the rest of the pipeline stays the same.
from llama_index.llms.openai_like import OpenAILike
from llama_index.core import Settings
# model is your ModelRiver workflow’s name.
Settings.llm = OpenAILike(
api_base="https://api.modelriver.com/v1",
api_key="mr_live_...",
model="docs-answer",
is_chat_model=True,
)
# Your index and query engine stay the same.
index.as_query_engine().query("How do I rotate an API key?")Backups built in
Each workflow has a primary model and up to 2 backups. Errors, timeouts and rate limits move the request to the next one.
Every request logged
Provider, model, tokens, cost, duration and every attempt are in the request logs.
Answers checked against a schema
Give a workflow a JSON schema and every response is checked against it. A mismatch goes to the next backup.
Repeat questions from cache
Turn on the cache for a workflow and identical requests are answered without calling the provider.
LlamaIndex vs ModelRiver
LlamaIndex works with your data. ModelRiver runs the model calls. Here is what each one gives you.
| Topic | LlamaIndex | ModelRiver |
|---|---|---|
| What it is | An open-source framework for loading, indexing and querying your data | A hosted AI backend you call over an API |
| Retrieval and indexing | Its core strength | Not included; keep LlamaIndex for this |
| Provider backups | Written in your code | Up to 2 backups per workflow |
| Request logs | Instrumentation plus a third-party tool | Built in, with every attempt |
| Structured output | Output parsers in your code | JSON Schema checked on every response |
| Streaming to the browser | You build the connection | Server-sent events, or WebSocket with the client SDK |
| Conversation memory | Chat memory in your code | Sessions kept by ModelRiver |
| Spending limits | Not included | Per project and per model |
Which one fits
Use LlamaIndex if
- You need custom chunking, indexing or retrieval strategies.
- You are building question answering over your own documents.
- You want fine control over how data is loaded and indexed.
Use ModelRiver if
- You want backups across providers without writing them.
- You want logs for every request, built in.
- You want answers checked against a schema, whichever model replies.
Most teams use both: LlamaIndex for retrieval, ModelRiver for the model calls.
Frequently asked questions
Is ModelRiver a full replacement for LlamaIndex?
No. LlamaIndex loads, indexes and retrieves your data. ModelRiver doesn’t do retrieval; it runs the model calls with backups, request logs, structured output and limits. Many teams use both.
How do I connect LlamaIndex to ModelRiver?
Use the OpenAILike LLM with the ModelRiver API address and your API key, and set model to a workflow name. Your index and query engines stay the same.
Can embeddings go through ModelRiver too?
Yes. Set up an embedding workflow and call the OpenAI-compatible embeddings endpoint with its name.
Does ModelRiver store my documents?
No. It only sees the prompts your pipeline sends, including the retrieved context. Request bodies are kept in the logs only if you turn on request body logging.
Docs and next reads
- LlamaIndex guideConnect LLMs and embeddings to ModelRiver. (opens in a new tab)
- Structured dataHow responses are checked against your schema. (opens in a new tab)
- CacheAnswer repeat questions without calling the provider. (opens in a new tab)
- Auto failoverHow a workflow moves to its backups when a provider fails. (opens in a new tab)
Keep LlamaIndex. Add the production layer.
Start on the free plan, create a workflow, and point your LlamaIndex LLM at it.