Request Types

Workflows now support different request types to handle various AI operations. The request_type field determines the expected message format and API endpoint usage.

Available Request Types

1. chat (Default)

Standard chat completions for conversational AI.

Use cases:

Conversational assistants
Q&A systems
Multi-turn dialogues
Chatbots

Required message format:

JSON

1{
2  "workflow": "my-chat-workflow",
3  "messages": [
4    {"role": "system", "content": "You are a helpful assistant."},
5    {"role": "user", "content": "Hello, how are you?"}
6  ],
7  "temperature": 0.7,
8  "max_tokens": 1000
9}

Supported providers:

OpenAI (gpt-4o, gpt-4-turbo, gpt-3.5-turbo, etc.)
Anthropic (claude-3-5-sonnet, claude-3-opus, etc.)

2. completion

Text completions for autocomplete and text generation.

Use cases:

Code completion
Text autocomplete
Prompt-based text generation
Single-shot text generation

Required message format:

JSON

1{
2  "workflow": "my-completion-workflow",
3  "prompt": "Once upon a time in a galaxy far far away",
4  "max_tokens": 500,
5  "temperature": 0.8
6}

Supported providers:

OpenAI (gpt-3.5-turbo-instruct, text-davinci-003, etc.)

3. image

Image generation from text descriptions.

Use cases:

Creating images from text prompts
Visual content generation
Design mockups
Concept art

Required message format:

JSON

1{
2  "workflow": "my-image-workflow",
3  "prompt": "A beautiful sunset over mountains with a lake in the foreground",
4  "size": "1024x1024",
5  "quality": "hd",
6  "n": 1
7}

Supported providers:

OpenAI (dall-e-3, dall-e-2)
Stability AI (stable-diffusion-xl, etc.)

Response format:

JSON

1{
2  "data": {
3    "created": 1234567890,
4    "data": [
5      {
6        "url": "https://...",
7        "revised_prompt": "..."
8      }
9    ]
10  }
11}

4. embedding

Generate vector embeddings for text.

Use cases:

Semantic search
Text similarity
Clustering
Recommendation systems
RAG (Retrieval Augmented Generation)

Required message format:

JSON

1{
2  "workflow": "my-embedding-workflow",
3  "input": "The quick brown fox jumps over the lazy dog",
4  "encoding_format": "float"
5}

Alternative (batch):

JSON

1{
2  "workflow": "my-embedding-workflow",
3  "input": [
4    "First text to embed",
5    "Second text to embed",
6    "Third text to embed"
7  ]
8}

Supported providers:

OpenAI (text-embedding-3-large, text-embedding-3-small, text-embedding-ada-002)
Cohere (embed-english-v3.0, embed-multilingual-v3.0)

Response format:

JSON

1{
2  "data": {
3    "object": "list",
4    "data": [
5      {
6        "object": "embedding",
7        "embedding": [0.123, -0.456, 0.789, ...],
8        "index": 0
9      }
10    ],
11    "model": "text-embedding-3-large",
12    "usage": {
13      "prompt_tokens": 8,
14      "total_tokens": 8
15    }
16  }
17}

5. audio

Audio transcription, translation, and generation.

Use cases:

Speech-to-text transcription
Audio translation
Text-to-speech
Voice generation

Required message format (transcription):

JSON

1{
2  "workflow": "my-audio-workflow",
3  "file": "base64_encoded_audio_data",
4  "model": "whisper-1",
5  "response_format": "json",
6  "language": "en"
7}

Required message format (text-to-speech):

JSON

1{
2  "workflow": "my-tts-workflow",
3  "input": "Hello, this is a test of text to speech.",
4  "voice": "alloy",
5  "response_format": "mp3"
6}

Supported providers:

OpenAI (whisper-1, tts-1, tts-1-hd)
ElevenLabs (various voices)

6. vision

Image analysis and understanding.

Use cases:

Image description
OCR (Optical Character Recognition)
Object detection
Visual question answering
Image classification

Required message format:

JSON

1{
2  "workflow": "my-vision-workflow",
3  "messages": [
4    {
5      "role": "user",
6      "content": [
7        {
8          "type": "text",
9          "text": "What's in this image?"
10        },
11        {
12          "type": "image_url",
13          "image_url": {
14            "url": "data:image/jpeg;base64,..."
15          }
16        }
17      ]
18    }
19  ]
20}

Alternative (URL-based):

JSON

1{
2  "workflow": "my-vision-workflow",
3  "messages": [
4    {
5      "role": "user",
6      "content": [
7        {"type": "text", "text": "Describe this image in detail"},
8        {
9          "type": "image_url",
10          "image_url": {
11            "url": "https://example.com/image.jpg",
12            "detail": "high"
13          }
14        }
15      ]
16    }
17  ]
18}

Supported providers:

OpenAI (gpt-4-vision-preview, gpt-4o)
Anthropic (claude-3-opus, claude-3-5-sonnet with vision)

Creating Workflows with Request Types

Dashboard API

Create a new workflow:

Bash

POST /api/console/projects/:project_id/workflows
Content-Type: application/json
 
{
  "name": "my_image_generator",
  "description": "Generate images from text descriptions",
  "provider": "openai",
  "model": "dall-e-3",
  "request_type": "image",
  "backup_1_provider": "stability",
  "backup_1_model": "stable-diffusion-xl"
}

Update an existing workflow:

Bash

PUT /api/console/projects/:project_id/workflows/:workflow_id
Content-Type: application/json
 
{
  "request_type": "vision",
  "model": "gpt-4o"
}

Request Type Validation

The system will validate that:

The request_type matches the expected format for the provider
The message structure conforms to the request type requirements
The selected model supports the specified request type

Example Error Response

JSON

1{
2  "error": "Invalid request format",
3  "message": "Request type 'image' requires 'prompt' field, not 'messages'",
4  "hint": "See docs/request-types.md for correct format"
5}

Default Behavior

If no request_type is specified when creating a workflow, it defaults to "chat".

Existing workflows without a request_type will automatically use "chat" to maintain backward compatibility.

Migration Guide

Updating Existing Workflows

If you have existing workflows that need to support different request types:

Identify the workflow:

Bash
GET /api/console/projects/:project_id/workflows
Update the request type:

Bash
PUT /api/console/projects/:project_id/workflows/:workflow_id { "request_type": "image" }
Test the workflow:

Bash
POST /api/console/projects/:project_id/workflows/test { "workflow_id": "...", "payload": { "prompt": "A beautiful landscape" } }

Best Practices

Match request type to use case: Choose the most appropriate request type for your application
Configure fallbacks: Set backup providers that support the same request type
Validate inputs: Ensure client applications send the correct message format
Use structured outputs: Combine request types with structured outputs for consistent responses
Monitor usage: Track which request types consume the most tokens/resources

Future Request Types

Planned support for:

function_calling - Function/tool calling
multimodal - Combined vision + text
fine_tuning - Fine-tuning jobs
moderation - Content moderation

For questions or feature requests, please open an issue on GitHub.