Request Types

Workflows now support different request types to handle various AI operations. The request_type field determines the expected message format and API endpoint usage.

Available Request Types

1. chat (Default)

Standard chat completions for conversational AI.

Use cases:

Conversational assistants
Q&A systems
Multi-turn dialogues
Chatbots

Required message format:

{
  "workflow": "my-chat-workflow",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello, how are you?"}
  ],
  "temperature": 0.7,
  "max_tokens": 1000
}

Supported providers:

OpenAI (gpt-4o, gpt-4-turbo, gpt-3.5-turbo, etc.)
Anthropic (claude-3-5-sonnet, claude-3-opus, etc.)

2. completion

Text completions for autocomplete and text generation.

Use cases:

Code completion
Text autocomplete
Prompt-based text generation
Single-shot text generation

Required message format:

{
  "workflow": "my-completion-workflow",
  "prompt": "Once upon a time in a galaxy far far away",
  "max_tokens": 500,
  "temperature": 0.8
}

Supported providers:

OpenAI (gpt-3.5-turbo-instruct, text-davinci-003, etc.)

3. image

Image generation from text descriptions.

Use cases:

Creating images from text prompts
Visual content generation
Design mockups
Concept art

Required message format:

{
  "workflow": "my-image-workflow",
  "prompt": "A beautiful sunset over mountains with a lake in the foreground",
  "size": "1024x1024",
  "quality": "hd",
  "n": 1
}

Supported providers:

OpenAI (dall-e-3, dall-e-2)
Stability AI (stable-diffusion-xl, etc.)

Response format:

{
  "data": {
    "created": 1234567890,
    "data": [
      {
        "url": "https://...",
        "revised_prompt": "..."
      }
    ]
  }
}

4. embedding

Generate vector embeddings for text.

Use cases:

Semantic search
Text similarity
Clustering
Recommendation systems
RAG (Retrieval Augmented Generation)

Required message format:

{
  "workflow": "my-embedding-workflow",
  "input": "The quick brown fox jumps over the lazy dog",
  "encoding_format": "float"
}

Alternative (batch):

{
  "workflow": "my-embedding-workflow",
  "input": [
    "First text to embed",
    "Second text to embed",
    "Third text to embed"
  ]
}

Supported providers:

OpenAI (text-embedding-3-large, text-embedding-3-small, text-embedding-ada-002)
Cohere (embed-english-v3.0, embed-multilingual-v3.0)

Response format:

{
  "data": {
    "object": "list",
    "data": [
      {
        "object": "embedding",
        "embedding": [0.123, -0.456, 0.789, ...],
        "index": 0
      }
    ],
    "model": "text-embedding-3-large",
    "usage": {
      "prompt_tokens": 8,
      "total_tokens": 8
    }
  }
}

5. audio

Audio transcription, translation, and generation.

Use cases:

Speech-to-text transcription
Audio translation
Text-to-speech
Voice generation

Required message format (transcription):

{
  "workflow": "my-audio-workflow",
  "file": "base64_encoded_audio_data",
  "model": "whisper-1",
  "response_format": "json",
  "language": "en"
}

Required message format (text-to-speech):

{
  "workflow": "my-tts-workflow",
  "input": "Hello, this is a test of text to speech.",
  "voice": "alloy",
  "response_format": "mp3"
}

Supported providers:

OpenAI (whisper-1, tts-1, tts-1-hd)
ElevenLabs (various voices)

6. vision

Image analysis and understanding.

Use cases:

Image description
OCR (Optical Character Recognition)
Object detection
Visual question answering
Image classification

Required message format:

{
  "workflow": "my-vision-workflow",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What's in this image?"
        },
        {
          "type": "image_url",
          "image_url": {
            "url": "data:image/jpeg;base64,..."
          }
        }
      ]
    }
  ]
}

Alternative (URL-based):

{
  "workflow": "my-vision-workflow",
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "Describe this image in detail"},
        {
          "type": "image_url",
          "image_url": {
            "url": "https://example.com/image.jpg",
            "detail": "high"
          }
        }
      ]
    }
  ]
}

Supported providers:

OpenAI (gpt-4-vision-preview, gpt-4o)
Anthropic (claude-3-opus, claude-3-5-sonnet with vision)

Creating Workflows with Request Types

Dashboard API

Create a new workflow:

POST /api/dashboard/projects/:project_id/workflows
Content-Type: application/json

{
  "name": "my_image_generator",
  "description": "Generate images from text descriptions",
  "provider": "openai",
  "model": "dall-e-3",
  "request_type": "image",
  "backup_1_provider": "stability",
  "backup_1_model": "stable-diffusion-xl"
}

Update an existing workflow:

PUT /api/dashboard/projects/:project_id/workflows/:workflow_id
Content-Type: application/json

{
  "request_type": "vision",
  "model": "gpt-4o"
}

Request Type Validation

The system will validate that:

The request_type matches the expected format for the provider
The message structure conforms to the request type requirements
The selected model supports the specified request type

Example Error Response

{
  "error": "Invalid request format",
  "message": "Request type 'image' requires 'prompt' field, not 'messages'",
  "hint": "See docs/REQUEST_TYPES.md for correct format"
}

Default Behavior

If no request_type is specified when creating a workflow, it defaults to "chat".

Existing workflows without a request_type will automatically use "chat" to maintain backward compatibility.

Migration Guide

Updating Existing Workflows

If you have existing workflows that need to support different request types:

Identify the workflow:

GET /api/dashboard/projects/:project_id/workflows

Update the request type:

PUT /api/dashboard/projects/:project_id/workflows/:workflow_id
{
  "request_type": "image"
}

Test the workflow:

POST /api/dashboard/projects/:project_id/workflows/test
{
  "workflow_id": "...",
  "payload": {
    "prompt": "A beautiful landscape"
  }
}

Best Practices

Match request type to use case: Choose the most appropriate request type for your application
Configure fallbacks: Set backup providers that support the same request type
Validate inputs: Ensure client applications send the correct message format
Use structured outputs: Combine request types with structured outputs for consistent responses
Monitor usage: Track which request types consume the most tokens/resources

Future Request Types

Planned support for:

function_calling - Function/tool calling
multimodal - Combined vision + text
fine_tuning - Fine-tuning jobs
moderation - Content moderation

For questions or feature requests, please open an issue on GitHub.