> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pipecat.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Frames

> Frame categories, processing behavior, and common patterns for Pipecat pipelines

## Overview

Frames are the fundamental units of data in Pipecat. Every piece of information that moves through a pipeline — audio, text, images, control signals — is wrapped in a frame. Frame processors receive frames, act on them, and push new or modified frames along to the next processor.

All frames inherit from the base `Frame` class and are Python [dataclasses](https://docs.python.org/3/library/dataclasses.html).

## Frame Categories

Pipecat has three base frame types, each with different processing behavior:

| Base Type      | Processing                                                    | Interruption Behavior                  |
| -------------- | ------------------------------------------------------------- | -------------------------------------- |
| `DataFrame`    | Queued, processed in order with non-SystemFrames              | Cancelled on user interruption         |
| `ControlFrame` | Queued, processed in order with non-SystemFrames              | Cancelled on user interruption         |
| `SystemFrame`  | Higher priority, queued, processed in order with SystemFrames | **Not** cancelled on user interruption |

### DataFrame

Data frames carry the main content flowing through a pipeline: audio chunks, text, images, and LLM messages. They are queued and processed in order with other DataFrames and ControlFrames. If a user interrupts (starts speaking while the bot is responding), any pending data frames are discarded so the new input can be handled immediately.

Examples: `TextFrame`, `OutputAudioRawFrame`, `LLMMessagesAppendFrame`, `TTSSpeakFrame`

### ControlFrame

ControlFrames signal processing boundaries and configuration changes: response start/end markers, settings updates, and state transitions. They are queued and processed in order alongside DataFrames, and like DataFrames, any pending ControlFrames are discarded when a user interrupts unless combined with `UninterruptibleFrame`.

Examples: `EndFrame`, `LLMFullResponseStartFrame`, `TTSStartedFrame`, `ServiceUpdateSettingsFrame`

### SystemFrame

SystemFrames are high-priority signals that must always be delivered: interruptions, user input, error notifications, and pipeline lifecycle events. They are queued and processed in order with other SystemFrames. Unlike DataFrames and ControlFrames, they are never discarded when a user interrupts.

Examples: `StartFrame`, `CancelFrame`, `InterruptionFrame.`, `UserStartedSpeakingFrame`, `InputAudioRawFrame`

## Frame Properties

Every frame has these properties set automatically:

<ParamField path="id" type="int">
  Unique identifier for the frame instance.
</ParamField>

<ParamField path="name" type="str">
  Human-readable name combining class name and instance count (e.g.,
  `TextFrame#3`). Useful for debugging.
</ParamField>

<ParamField path="pts" type="int | None">
  Presentation timestamp in nanoseconds. Used for audio/video synchronization.
</ParamField>

<ParamField path="broadcast_sibling_id" type="int | None">
  ID of the paired frame when a frame is broadcast in both directions. Set
  automatically by `broadcast_frame()` and `broadcast_frame_instance()`.
</ParamField>

<ParamField path="metadata" type="Dict[str, Any]">
  Dictionary for arbitrary frame metadata.
</ParamField>

<ParamField path="transport_source" type="str | None">
  Name of the transport source that created this frame.
</ParamField>

<ParamField path="transport_destination" type="str | None">
  Name of the transport destination for this frame. Used when a transport
  supports multiple output tracks.
</ParamField>

## Frame Direction

Frames flow through the pipeline in one of two directions:

```python theme={null}
from pipecat.processors.frame_processor import FrameDirection

class FrameDirection(Enum):
    DOWNSTREAM = 1  # Input → Output (default)
    UPSTREAM = 2    # Output → Input
```

**Downstream** is the default. In a typical voice AI pipeline, audio enters from the transport input, gets transcribed, runs through the LLM, converts to speech, and reaches the transport output.

**Upstream** lets processors send information back toward the start of the pipeline. The most common example: the assistant context aggregator at the end of the pipeline pushes context frames upstream so they flow back to the LLM.

### Pushing Frames

Within a frame processor, call `push_frame()` to send a frame to the next processor:

```python theme={null}
# Push downstream (default)
await self.push_frame(frame, FrameDirection.DOWNSTREAM)

# Push upstream
await self.push_frame(frame, FrameDirection.UPSTREAM)
```

### Broadcasting Frames

To send a frame in **both** directions simultaneously, use `broadcast_frame()`:

```python theme={null}
# Create and push instances upstream and downstream
await self.broadcast_frame(UserStartedSpeakingFrame)
```

Each direction receives its own frame instance, linked by `broadcast_sibling_id`.

To broadcast an existing frame instance (when you are not the original creator of the frame), use `broadcast_frame_instance()`:

```python theme={null}
# Broadcast an existing frame instance in both directions
await self.broadcast_frame_instance(frame)
```

This creates two new instances by shallow-copying all fields from the original frame except `id` and `name`, which get fresh values.

<Tip>Prefer `broadcast_frame()` when possible, as it is more efficient.</Tip>

## Mixins

Mixins add cross-cutting behavior or shared data fields to frames without changing their base type.

### UninterruptibleFrame

Occasionally a `DataFrame` or `ControlFrame` is too important to discard during an interruption. Adding the `UninterruptibleFrame` mixin protects it: the frame stays in internal queues and any task processing it will not be cancelled.

```python theme={null}
@dataclass
class FunctionCallResultFrame(DataFrame, UninterruptibleFrame):
    """Must be delivered even if the user interrupts."""
    ...
```

Examples: `EndFrame`, `StopFrame`, `FunctionCallResultFrame`, `FunctionCallInProgressFrame`

### AudioRawFrame

Carries raw audio fields shared by both input and output audio frames.

<ParamField path="audio" type="bytes">
  Raw audio bytes in PCM format.
</ParamField>

<ParamField path="sample_rate" type="int">
  Audio sample rate in Hz (e.g., 16000).
</ParamField>

<ParamField path="num_channels" type="int">
  Number of audio channels (e.g., 1 for mono).
</ParamField>

<ParamField path="num_frames" type="int">
  Number of audio frames. Calculated automatically from the audio data.
</ParamField>

### ImageRawFrame

Carries raw image fields shared by both input and output image frames.

<ParamField path="image" type="bytes">
  Raw image bytes.
</ParamField>

<ParamField path="size" type="Tuple[int, int]">
  Image dimensions as (width, height).
</ParamField>

<ParamField path="format" type="str | None">
  Image format (e.g., `"RGB"`, `"RGBA"`).
</ParamField>

## Common Patterns

Pipecat prefers pushing frames over calling methods directly between processors. Routing data through the pipeline as frames ensures correct processing order, which is critical for real-time use cases.

Most frames are produced and consumed by Pipecat's built-in services. The patterns below cover the frames you're most likely to push yourself in application code.

### Starting a Conversation

Add an initial message to the context, then push `LLMRunFrame` to kick off processing:

```python theme={null}
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
    context.add_message({"role": "user", "content": "Please introduce yourself."})
    await task.queue_frames([LLMRunFrame()])
```

### Injecting a Prompt

`LLMMessagesAppendFrame` adds messages to the context without replacing what's already there. Set `run_llm=True` to trigger a response immediately:

```python theme={null}
message = {
    "role": "user",
    "content": "The user has been quiet. Ask if they're still there.",
}
await aggregator.push_frame(LLMMessagesAppendFrame([message], run_llm=True))
```

### Speaking Without the LLM

`TTSSpeakFrame` sends text directly to the TTS service as a standalone utterance, bypassing the LLM entirely:

```python theme={null}
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
    await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
```

### Ending a Conversation

Push `EndTaskFrame` upstream to gracefully shut down the pipeline. Pair it with a `TTSSpeakFrame` to say goodbye first:

```python theme={null}
await aggregator.push_frame(
    TTSSpeakFrame("It seems like you're busy. Have a nice day!")
)
await aggregator.push_frame(EndTaskFrame(), FrameDirection.UPSTREAM)
```

### Changing Service Settings at Runtime

Push settings frames to adjust LLM, TTS, or STT configuration mid-conversation:

```python theme={null}
await task.queue_frame(
    LLMUpdateSettingsFrame(delta=OpenAILLMService.Settings(temperature=0.1))
)
```

### Updating Tools at Runtime

Add or replace available function-calling tools while the conversation is active:

```python theme={null}
new_tools = ToolsSchema(
    standard_tools=[weather_function, restaurant_function]
)
await task.queue_frames([LLMSetToolsFrame(tools=new_tools)])
```

### Playing Sound Effects

Load audio files and push `OutputAudioRawFrame` directly from a custom processor:

```python theme={null}
with wave.open("ding.wav") as f:
    ding = OutputAudioRawFrame(f.readframes(-1), f.getframerate(), f.getnchannels())

class SoundEffect(FrameProcessor):
    async def process_frame(self, frame, direction):
        await super().process_frame(frame, direction)
        if isinstance(frame, LLMFullResponseEndFrame):
            await self.push_frame(ding)
        await self.push_frame(frame, direction)
```

### Reacting to LLM Response Boundaries

`LLMFullResponseStartFrame` and `LLMFullResponseEndFrame` bracket every LLM response. Custom processors can watch for these to trigger side effects:

```python theme={null}
class ResponseLogger(FrameProcessor):
    async def process_frame(self, frame, direction):
        await super().process_frame(frame, direction)
        if isinstance(frame, LLMFullResponseStartFrame):
            logger.info("LLM response started")
        elif isinstance(frame, LLMFullResponseEndFrame):
            logger.info("LLM response finished")
        await self.push_frame(frame, direction)
```

## Frame Type Reference

The individual reference pages below document every frame class, organized by function:

<CardGroup cols={2}>
  <Card title="Data Frames" icon="database" href="/api-reference/server/frames/data-frames">
    Audio, image, text, transcription, and transport message frames that carry
    content through the pipeline.
  </Card>

  <Card title="Control Frames" icon="sliders" href="/api-reference/server/frames/control-frames">
    Pipeline lifecycle, LLM response boundaries, TTS state, service settings,
    and filter/mixer configuration.
  </Card>

  <Card title="System Frames" icon="bolt" href="/api-reference/server/frames/system-frames">
    Interruptions, user/bot speaking state, VAD events, errors, metrics, and raw
    input frames.
  </Card>

  <Card title="LLM Frames" icon="brain" href="/api-reference/server/frames/llm-frames">
    LLM context frame, function calling helper dataclasses, and links to
    LLM-related frames on other pages.
  </Card>
</CardGroup>