Separating Environments – ModelRiver Docs

Overview

When production logs are mixed with test and playground data, it becomes nearly impossible to assess the true health of your AI application. ModelRiver automatically tags every request with its source environment, letting you filter cleanly and keep your analysis accurate.

How environment tagging works

Every request log records where the request came from (your API keys or the console playground) and whether its workflow ran in test mode. The Source filter in Request Logs is built on this:

Source filter	What it includes
Production	Requests sent with your API keys
Playground	Requests run from the console playground
Cache	Responses served from cache
Test	Requests to workflows in test mode, from the API or the playground

Some requests also carry a seed_batch prefix, such as test_mode: for test mode calls to /v1/ai, pg: and pg_test_mode: for playground runs, and pg_callback: for simulated playground callbacks. See Using filters for the full list.

Best practices for environment separation

1. Always filter before analyzing

Debugging production issues?   → Source "Production"
Reviewing test results?        → Source "Test"
Validating a workflow change?  → Source "Playground" (+ Revision "Draft")
Looking at everything?         → No filters (rare)

2. Use test mode for integration tests

Test mode is a workflow setting, not a request flag. When running automated tests or CI/CD pipelines, send requests to a workflow whose mode is set to Testing (for example, a test copy of your production workflow):

Bash

# The workflow "support_summarizer_test" is in test mode
curl -X POST https://api.modelriver.com/v1/chat/completions \
  -H "Authorization: Bearer $MODELRIVER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "support_summarizer_test",
    "messages": [{"role": "user", "content": "test"}]
  }'

This ensures test traffic:

Uses sample data instead of real providers (free)
Is logged as test mode and appears under the Test source filter
Isn't billed and isn't counted against spending limits

Because these requests are sent with your API keys, they also appear under Production. To keep CI traffic out of production views entirely, run it in a separate project.

3. Use playground for manual validation

Before deploying changes:

Test mode first: Verify workflow structure (free, sample data)
Production mode: Validate with real AI responses (uses credits)
Deploy: Push to production
Monitor production: Filter Source to Production and watch for any issues

Use the Source and Revision filters to find the logs from each step.

4. Never analyze unfiltered logs for health metrics

When calculating error rates, latency, or costs:

✗ Error rate across all requests = 5.2%
  (Includes failed tests, which inflates the number)
 
✓ Error rate with Source "Production" = 1.8%
  (Actual production health)

Using all requests for metrics gives misleading results because test failures and playground experiments are counted alongside real production traffic.

Environment separation checklist

All CI/CD test calls go to workflows in test mode (or to a separate project)
Manual testing uses the Playground, not the production API
Health monitoring filters Source to "Production"
Cost analysis filters Source to "Production" for accurate spending data
Debugging starts with the correct filter for the issue context

Next steps

Token Optimization: Optimize costs
Using Filters Effectively: Master the filter system
Back to Best Practices: Return to the overview

Keep your environments cleanly separated