> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pipecat.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Frames

> Frame categories, processing behavior, and common patterns for Pipecat pipelines

## Overview

Frames are the fundamental units of data in Pipecat. Every piece of information that moves through a pipeline — audio, text, images, control signals — is wrapped in a frame. Frame processors receive frames, act on them, and push new or modified frames along to the next processor.

All frames inherit from the base `Frame` class and are Python [dataclasses](https://docs.python.org/3/library/dataclasses.html).

## Frame Categories

Pipecat has three base frame types, each with different processing behavior:

| Base Type | Processing | Interruption Behavior |
| - | - | - |
| `DataFrame` | Queued, processed in order with non-SystemFrames | Cancelled on user interruption |
| `ControlFrame` | Queued, processed in order with non-SystemFrames | Cancelled on user interruption |
| `SystemFrame` | Higher priority, queued, processed in order with SystemFrames | **Not** cancelled on user interruption |

### DataFrame

Data frames carry the main content flowing through a pipeline: audio chunks, text, images, and LLM messages. They are queued and processed in order with other DataFrames and ControlFrames. If a user interrupts (starts speaking while the bot is responding), any pending data frames are discarded so the new input can be handled immediately.

Examples: `TextFrame`, `OutputAudioRawFrame`, `LLMMessagesAppendFrame`, `TTSSpeakFrame`

### ControlFrame

ControlFrames signal processing boundaries and configuration changes: response start/end markers, settings updates, and state transitions. They are queued and processed in order alongside DataFrames, and like DataFrames, any pending ControlFrames are discarded when a user interrupts unless their `interruptible` field is `False`.

Examples: `EndFrame`, `LLMFullResponseStartFrame`, `TTSStartedFrame`, `ServiceUpdateSettingsFrame`

### SystemFrame

SystemFrames are high-priority signals that must always be delivered: interruptions, user input, error notifications, and pipeline lifecycle events. They are queued and processed in order with other SystemFrames. Unlike DataFrames and ControlFrames, they are never discarded when a user interrupts.

Examples: `StartFrame`, `CancelFrame`, `InterruptionFrame.`, `UserStartedSpeakingFrame`, `InputAudioRawFrame`

## Frame Properties

Every frame has these properties set automatically:

**id** `int`

Unique identifier for the frame instance.

**name** `str`

Human-readable name combining class name and instance count (e.g.,
`TextFrame#3`). Useful for debugging.

**pts** `int | None`

Presentation timestamp in nanoseconds. Used for audio/video synchronization.

**broadcast\_sibling\_id** `int | None`

ID of the paired frame when a frame is broadcast in both directions. Set
automatically by `broadcast_frame()` and `broadcast_frame_instance()`.

**metadata** `Dict[str, Any]`

Dictionary for arbitrary frame metadata.

**transport\_source** `str | None`

Name of the transport source that created this frame.

**transport\_destination** `str | None`

Name of the transport destination for this frame. Used when a transport
supports multiple output tracks.

**interruptible** `bool` (default: True)

Whether an interruption may drop this frame from a processor's queue or cancel
its processing. True by default. A frame class such as `EndFrame` declares
`interruptible: bool = field(default=False, init=False)` to be uninterruptible
by default. Set it on a frame before pushing it to decide for that frame
alone.

## Frame Direction

Frames flow through the pipeline in one of two directions:

```python theme={null}
from pipecat.processors.frame_processor import FrameDirection

class FrameDirection(Enum):
    DOWNSTREAM = 1  # Input → Output (default)
    UPSTREAM = 2    # Output → Input
```

**Downstream** is the default. In a typical voice AI pipeline, audio enters from the transport input, gets transcribed, runs through the LLM, converts to speech, and reaches the transport output.

**Upstream** lets processors send information back toward the start of the pipeline. The most common example: the assistant context aggregator at the end of the pipeline pushes context frames upstream so they flow back to the LLM.

### Pushing Frames

Within a frame processor, call `push_frame()` to send a frame to the next processor:

```python theme={null}
# Push downstream (default)
await self.push_frame(frame, FrameDirection.DOWNSTREAM)

# Push upstream
await self.push_frame(frame, FrameDirection.UPSTREAM)
```

### Broadcasting Frames

To send a frame in **both** directions simultaneously, use `broadcast_frame()`:

```python theme={null}
# Create and push instances upstream and downstream
await self.broadcast_frame(UserStartedSpeakingFrame)
```

Each direction receives its own frame instance, linked by `broadcast_sibling_id`.

To broadcast an existing frame instance (when you are not the original creator of the frame), use `broadcast_frame_instance()`:

```python theme={null}
# Broadcast an existing frame instance in both directions
await self.broadcast_frame_instance(frame)
```

This creates two new instances by shallow-copying all fields from the original frame except `id` and `name`, which get fresh values.

> [!TIP]
>
> Prefer `broadcast_frame()` when possible, as it is more efficient.

## Mixins

Mixins add cross-cutting behavior or shared data fields to frames without changing their base type.

### UninterruptibleFrame

> [!WARNING]
>
> **Deprecated.** Use the `interruptible` field with `default=False` instead.
> `UninterruptibleFrame` will be removed in 2.0.0.

Occasionally a `DataFrame` or `ControlFrame` is too important to discard during an interruption. To protect a frame, declare the `interruptible` field with `default=False` in the frame class:

```python theme={null}
from dataclasses import dataclass, field

@dataclass
class FunctionCallResultFrame(DataFrame):
    """Must be delivered even if the user interrupts."""
    interruptible: bool = field(default=False, init=False)
    # ... other fields
```

You can also set the field on an individual frame to override the class default:

```python theme={null}
frame = TextFrame(text="Important")
frame.interruptible = False  # This specific frame survives interruption
await self.push_frame(frame)
```

Examples of uninterruptible frames: `EndFrame`, `StopFrame`, `FunctionCallResultFrame`, `FunctionCallInProgressFrame`

### AudioRawFrame

Carries raw audio fields shared by both input and output audio frames.

**audio** `bytes`

Raw audio bytes in PCM format.

**sample\_rate** `int`

Audio sample rate in Hz (e.g., 16000).

**num\_channels** `int`

Number of audio channels (e.g., 1 for mono).

**num\_frames** `int`

Number of audio frames. Calculated automatically from the audio data.

### ImageRawFrame

Carries raw image fields shared by both input and output image frames.

**image** `bytes`

Raw image bytes.

**size** `Tuple[int, int]`

Image dimensions as (width, height).

**format** `str | None`

Image format (e.g., `"RGB"`, `"RGBA"`).

### FileRawFrame

Carries file fields shared by input file frames.

**file** `str`

A base64 data URL (`data:<mime>;base64,...`) or a URL string, depending on `type`.

**type** `Literal['bytes', 'url']`

Whether `file` holds the file bytes as a data URL (`"bytes"`) or a URL
(`"url"`).

**filename** `str | None`

Optional name of the file.

**format** `str`

MIME type of the file (e.g., `"application/pdf"`).

## Creating a Custom Frame

Frames are plain Python dataclasses, so you can define your own by subclassing one of the three base categories and decorating it with `@dataclass`. Pick the base class by how the frame should be handled:

* `DataFrame` for ordered data such as text, audio, or your own payload. Cancelled by user interruptions.
* `ControlFrame` for ordered control signals that flush in line with the data.
* `SystemFrame` for high-priority signals that skip the queue and are not affected by interruptions.

```python theme={null}
from dataclasses import dataclass

from pipecat.frames.frames import DataFrame


@dataclass
class AsteriskCommandFrame(DataFrame):
    """A custom command sent downstream to a telephony transport."""

    command: str
```

Your frame travels the pipeline like any built-in frame, in the direction you push it (see [Frame Direction](#frame-direction)). If the frame must survive a user interruption, declare `interruptible` with `default=False`:

```python theme={null}
from dataclasses import dataclass, field

@dataclass
class AsteriskCommandFrame(DataFrame):
    """A custom command sent downstream to a telephony transport."""

    interruptible: bool = field(default=False, init=False)
    command: str
```

## Common Patterns

Pipecat prefers pushing frames over calling methods directly between processors. Routing data through the pipeline as frames ensures correct processing order, which is critical for real-time use cases.

Most frames are produced and consumed by Pipecat's built-in services. The patterns below cover the frames you're most likely to push yourself in application code.

### Starting a Conversation

Add an initial message to the context, then push `LLMRunFrame` to kick off processing:

```python theme={null}
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
    context.add_message({"role": "user", "content": "Please introduce yourself."})
    await worker.queue_frames([LLMRunFrame()])
```

### Injecting a Prompt

`LLMMessagesAppendFrame` adds messages to the context without replacing what's already there. Set `run_llm=True` to trigger a response immediately:

```python theme={null}
message = {
    "role": "user",
    "content": "The user has been quiet. Ask if they're still there.",
}
await aggregator.push_frame(LLMMessagesAppendFrame([message], run_llm=True))
```

### Speaking Without the LLM

`TTSSpeakFrame` sends text directly to the TTS service as a standalone utterance, bypassing the LLM entirely:

```python theme={null}
@llm.event_handler("on_function_calls_started")
async def on_function_calls_started(service, function_calls):
    await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
```

### Ending a Conversation

Push `EndWorkerFrame` **downstream** to gracefully shut down the pipeline. Pair it with a `TTSSpeakFrame` to say goodbye first:

```python theme={null}
await aggregator.push_frame(
    TTSSpeakFrame("It seems like you're busy. Have a nice day!")
)
await aggregator.push_frame(EndWorkerFrame(), FrameDirection.DOWNSTREAM)
```

Downstream is `push_frame`'s default direction, so `await aggregator.push_frame(EndWorkerFrame())` is equivalent. Pushing it downstream is what lets the goodbye reach the TTS service and the transport before the pipeline ends — the frame queues behind that work instead of jumping ahead of it.

> [!TIP]
>
> See [Pipeline Termination](https://docs.pipecat.ai/pipecat/learn/pipeline-termination.md) for the
> difference between `EndFrame` and `EndWorkerFrame`, [why the direction
> matters](https://docs.pipecat.ai/pipecat/learn/pipeline-termination.md#incorrect-termination-frame-direction),
> and how to end a conversation from inside a tool call.

### Changing Service Settings at Runtime

Push settings frames to adjust LLM, TTS, or STT configuration mid-conversation:

```python theme={null}
await worker.queue_frame(
    LLMUpdateSettingsFrame(delta=OpenAILLMService.Settings(temperature=0.1))
)
```

### Updating Tools at Runtime

Add or replace available function-calling tools while the conversation is active:

```python theme={null}
new_tools = ToolsSchema(
    standard_tools=[weather_function, restaurant_function]
)
await worker.queue_frames([LLMSetToolsFrame(tools=new_tools)])
```

### Playing Sound Effects

Load audio files and push `OutputAudioRawFrame` directly from a custom processor:

```python theme={null}
with wave.open("ding.wav") as f:
    ding = OutputAudioRawFrame(f.readframes(-1), f.getframerate(), f.getnchannels())

class SoundEffect(FrameProcessor):
    async def process_frame(self, frame, direction):
        await super().process_frame(frame, direction)
        if isinstance(frame, LLMFullResponseEndFrame):
            await self.push_frame(ding)
        await self.push_frame(frame, direction)
```

### Reacting to LLM Response Boundaries

`LLMFullResponseStartFrame` and `LLMFullResponseEndFrame` bracket every LLM response. Custom processors can watch for these to trigger side effects:

```python theme={null}
class ResponseLogger(FrameProcessor):
    async def process_frame(self, frame, direction):
        await super().process_frame(frame, direction)
        if isinstance(frame, LLMFullResponseStartFrame):
            logger.info("LLM response started")
        elif isinstance(frame, LLMFullResponseEndFrame):
            logger.info("LLM response finished")
        await self.push_frame(frame, direction)
```

## Frame Type Reference

The individual reference pages below document every frame class, organized by function:

> [card] [Data Frames](https://docs.pipecat.ai/api-reference/server/frames/data-frames.md)
>
> Audio, image, text, transcription, and transport message frames that carry
> content through the pipeline.

> [card] [Control Frames](https://docs.pipecat.ai/api-reference/server/frames/control-frames.md)
>
> Pipeline lifecycle, LLM response boundaries, TTS state, service settings,
> and filter/mixer configuration.

> [card] [System Frames](https://docs.pipecat.ai/api-reference/server/frames/system-frames.md)
>
> Interruptions, user/bot speaking state, VAD events, errors, metrics, and raw
> input frames.

> [card] [LLM Frames](https://docs.pipecat.ai/api-reference/server/frames/llm-frames.md)
>
> LLM context frame, function calling helper dataclasses, and links to
> LLM-related frames on other pages.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.