Docs

Documentation

Request types for every AI operation

ModelRiver workflows support different request types: chat, completion, image, embedding, audio, and vision: each with tailored message formats and provider support.

Workflows support different request types to handle various AI operations. The request_type field determines the expected message format and API endpoint usage.

Available request types

Chat (default)

Standard chat completions for conversational AI.

Use cases: Conversational assistants, Q&A systems, multi-turn dialogues, chatbots

JSON
1{
2 "workflow": "my-chat-workflow",
3 "messages": [
4 {"role": "system", "content": "You are a helpful assistant."},
5 {"role": "user", "content": "Hello, how are you?"}
6 ],
7 "temperature": 0.7,
8 "max_tokens": 1000
9}

Supported providers:

  • OpenAI (gpt-4o, gpt-4-turbo, gpt-3.5-turbo, etc.)
  • Anthropic (claude-3-5-sonnet, claude-3-opus, etc.)
  • xAI (Grok)
  • Mistral (Mistral Large, Mixtral)
  • Google (Gemini)

Completion

Text completions for autocomplete and single-shot text generation.

Use cases: Code completion, text autocomplete, prompt-based text generation

JSON
1{
2 "workflow": "my-completion-workflow",
3 "prompt": "Once upon a time in a galaxy far far away",
4 "max_tokens": 500,
5 "temperature": 0.8
6}

Supported providers:

  • OpenAI (gpt-3.5-turbo-instruct, text-davinci-003, etc.)

Image

Image generation from text descriptions.

Use cases: Creating images from text prompts, visual content generation, design mockups, concept art

JSON
1{
2 "workflow": "my-image-workflow",
3 "prompt": "A beautiful sunset over mountains with a lake in the foreground",
4 "size": "1024x1024",
5 "quality": "hd",
6 "n": 1
7}

Supported providers:

  • OpenAI (dall-e-3, dall-e-2)
  • Stability AI (stable-diffusion-xl, etc.)

Response:

JSON
1{
2 "data": {
3 "created": 1234567890,
4 "data": [
5 {
6 "url": "https://...",
7 "revised_prompt": "..."
8 }
9 ]
10 }
11}

Embedding

Generate vector embeddings for text: essential for semantic search, RAG, and clustering.

Use cases: Semantic search, text similarity, clustering, recommendation systems, RAG (Retrieval Augmented Generation)

Single text:

JSON
1{
2 "workflow": "my-embedding-workflow",
3 "input": "The quick brown fox jumps over the lazy dog",
4 "encoding_format": "float"
5}

Batch (multiple texts):

JSON
1{
2 "workflow": "my-embedding-workflow",
3 "input": [
4 "First text to embed",
5 "Second text to embed",
6 "Third text to embed"
7 ]
8}

Supported providers:

  • OpenAI (text-embedding-3-large, text-embedding-3-small, text-embedding-ada-002)
  • Cohere (embed-english-v3.0, embed-multilingual-v3.0)

Response:

JSON
1{
2 "data": {
3 "object": "list",
4 "data": [
5 {
6 "object": "embedding",
7 "embedding": [0.123, -0.456, 0.789],
8 "index": 0
9 }
10 ],
11 "model": "text-embedding-3-large",
12 "usage": {
13 "prompt_tokens": 8,
14 "total_tokens": 8
15 }
16 }
17}

Audio

Audio transcription, translation, and text-to-speech generation.

Use cases: Speech-to-text transcription, audio translation, text-to-speech, voice generation

Transcription:

JSON
1{
2 "workflow": "my-audio-workflow",
3 "file": "base64_encoded_audio_data",
4 "model": "whisper-1",
5 "response_format": "json",
6 "language": "en"
7}

Text-to-speech:

JSON
1{
2 "workflow": "my-tts-workflow",
3 "input": "Hello, this is a test of text to speech.",
4 "voice": "alloy",
5 "response_format": "mp3"
6}

Supported providers:

  • OpenAI (whisper-1, tts-1, tts-1-hd)
  • ElevenLabs (various voices)

Vision

Image analysis and understanding.

Use cases: Image description, OCR, object detection, visual question answering, image classification

Base64 image:

JSON
1{
2 "workflow": "my-vision-workflow",
3 "messages": [
4 {
5 "role": "user",
6 "content": [
7 {"type": "text", "text": "What's in this image?"},
8 {
9 "type": "image_url",
10 "image_url": {"url": "data:image/jpeg;base64,..."}
11 }
12 ]
13 }
14 ]
15}

URL-based image:

JSON
1{
2 "workflow": "my-vision-workflow",
3 "messages": [
4 {
5 "role": "user",
6 "content": [
7 {"type": "text", "text": "Describe this image in detail"},
8 {
9 "type": "image_url",
10 "image_url": {
11 "url": "https://example.com/image.jpg",
12 "detail": "high"
13 }
14 }
15 ]
16 }
17 ]
18}

Supported providers:

  • OpenAI (gpt-4-vision-preview, gpt-4o)
  • Anthropic (claude-3-opus, claude-3-5-sonnet with vision)

Creating workflows with request types

Via the console

  1. Open Workflows in your project
  2. Click Create Workflow
  3. Select the Request Type from the dropdown
  4. Choose a provider and model that supports your selected type
  5. Save the workflow

Via the API

Bash
POST /api/console/projects/:project_id/workflows
Content-Type: application/json
 
{
"name": "my_image_generator",
"description": "Generate images from text descriptions",
"provider": "openai",
"model": "dall-e-3",
"request_type": "image",
"backup_1_provider": "stability",
"backup_1_model": "stable-diffusion-xl"
}

Validation

The system validates that:

  1. The request_type matches the expected format for the provider
  2. The message structure conforms to the request type requirements
  3. The selected model supports the specified request type

Example error:

JSON
1{
2 "error": "Invalid request format",
3 "message": "Request type 'image' requires 'prompt' field, not 'messages'",
4 "hint": "See /docs/api/request-types for correct format"
5}

Default behaviour

If no request_type is specified when creating a workflow, it defaults to "chat". Existing workflows without a request_type automatically use "chat" for backward compatibility.

Quick reference

TypeKey fieldTypical models
chatmessagesGPT-4o, Claude 3.5, Gemini
completionpromptGPT-3.5-instruct
imageprompt + sizeDALL-E 3, Stable Diffusion
embeddinginputtext-embedding-3-large
audiofile or inputWhisper, TTS-1
visionmessages (with image)GPT-4o, Claude 3.5

Best practices

  1. Match request type to use case: Choose the most appropriate type for your application
  2. Configure fallbacks: Set backup providers that support the same request type
  3. Validate inputs: Ensure application sends the correct message format for the type
  4. Use structured outputs: Combine request types with structured outputs for consistent responses
  5. Monitor usage: Track which request types consume the most tokens/resources

Next steps