> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pipecat.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI GPT-Live

> OpenAILiveLLMService connects Pipecat to OpenAI's GPT-Live API for full-duplex speech-to-speech conversation with delegated tools.

## Overview

`OpenAILiveLLMService` provides full-duplex speech-to-speech conversation using OpenAI's GPT-Live API and the `gpt-live-1` model. The GPT-Live model listens and speaks at the same time: it decides on its own when to answer, stops when the user talks over it, and delegates work that needs search, reasoning, or tools to a backend text model while the conversation continues. The pipeline streams audio in and plays audio out; there is no client-side turn detection or response triggering.

The backend runs in one of two **delegation modes**. In **Responses delegation**, OpenAI hosts the backend: a Responses API model you configure with `OpenAILiveLLMService.ResponsesDelegation`, using the tools from your pipeline's `LLMContext` as normal. In **client delegation**, the backend is any Pipecat LLM service running in a `BackendLLMWorker` you provide, configured with `OpenAILiveLLMService.ClientDelegation`, with tools of its own.

<CardGroup cols={2}>
  <Card title="OpenAI GPT-Live API Reference" icon="code" href="https://reference-server.pipecat.ai/en/latest/api/pipecat.services.openai.live.llm.html">
    Pipecat's API methods for GPT-Live integration
  </Card>

  <Card title="Example Implementation" icon="play" href="https://github.com/pipecat-ai/pipecat/blob/main/examples/realtime/realtime-openai-live-responses-delegation.py">
    Complete GPT-Live conversation example with Responses delegation
  </Card>

  <Card title="Client Delegation Example" icon="play" href="https://github.com/pipecat-ai/pipecat/blob/main/examples/realtime/realtime-openai-live-client-delegation.py">
    Delegating to a Pipecat backend worker running another LLM
  </Card>

  <Card title="OpenAI Documentation" icon="book" href="https://developers.openai.com/api/docs/guides/live">
    Official GPT-Live documentation
  </Card>

  <Card title="OpenAI Platform" icon="external-link" href="https://platform.openai.com/">
    Access GPT-Live models and manage API keys
  </Card>
</CardGroup>

## Installation

To use GPT-Live, install the required dependencies:

```bash theme={null}
uv add "pipecat-ai[openai]"
```

## Prerequisites

### OpenAI Account Setup

Before using GPT-Live, you need:

1. **OpenAI Account**: Sign up at [OpenAI Platform](https://platform.openai.com/)
2. **API Key**: Generate an OpenAI API key from your account dashboard
3. **Model Access**: Ensure access to the `gpt-live-1` model
4. **Usage Limits**: Configure appropriate usage limits and billing

### Required Environment Variables

* `OPENAI_API_KEY`: Your OpenAI API key for authentication

### Key Features

* **Full-Duplex Speech-to-Speech**: The model listens while it speaks and handles interruptions itself
* **Delegation**: Tool use and heavier reasoning run on a backend model while the conversation continues
* **Two Backend Modes**: Responses delegation, where OpenAI hosts the backend model, or client delegation, where any Pipecat LLM service runs as the backend in a `BackendLLMWorker`
* **Function Calling**: In Responses delegation mode, the functions registered in your pipeline, written as for any other LLM service; in client delegation mode, the functions registered in the backend worker's pipeline
* **Conversation Seeding**: Prior conversation history is loaded into the session at start

## Configuration

### OpenAILiveLLMService

<ParamField path="api_key" type="str" required>
  OpenAI project API key for authentication.
</ParamField>

<ParamField path="base_url" type="str" default="wss://api.openai.com/v1/live/sessions">
  WebSocket base URL of the GPT-Live API. Override for custom or proxied
  deployments.
</ParamField>

<ParamField path="settings" type="OpenAILiveLLMService.Settings" default="None">
  Runtime-updatable settings. See [Settings](#settings) below.
</ParamField>

<ParamField path="delegation" type="ResponsesDelegation | ClientDelegation | None" default="None">
  Where the model's delegated work runs. See
  [ResponsesDelegation](#responsesdelegation) and
  [ClientDelegation](#clientdelegation) below. `None` selects client delegation
  with no backend configured, so delegated requests are declined.
</ParamField>

<ParamField path="**kwargs" type="Any">
  Additional arguments passed to parent LLMService.
</ParamField>

### Settings

Settings passed via the `settings` constructor argument using `OpenAILiveLLMService.Settings(...)`. See [Service Settings](/pipecat/fundamentals/service-settings) for details.

| Parameter            | Type  | Default     | Description                                                                                  |
| -------------------- | ----- | ----------- | -------------------------------------------------------------------------------------------- |
| `model`              | `str` | `NOT_GIVEN` | GPT-Live model identifier. *(Inherited from base settings.)*                                 |
| `system_instruction` | `str` | `NOT_GIVEN` | Instructions for the GPT-Live model. *(Inherited from base settings.)*                       |
| `voice`              | `str` | `NOT_GIVEN` | Output voice name (for example `"marin"` or `"cedar"`). `None` leaves the choice to the API. |

<Note>
  `NOT_GIVEN` values are omitted, letting the service use its own defaults
  (`"gpt-live-1"` for model). The API fixes `model`, `voice` and
  `system_instruction` when the session starts, so a change sent with
  `LLMUpdateSettingsFrame` is stored, logged as unsupported for the session in
  progress, and applied to the next session started by `reset_conversation()`.
</Note>

### ResponsesDelegation

OpenAI hosts the backend model. Passed as `delegation=OpenAILiveLLMService.ResponsesDelegation(...)`.

<ParamField path="settings" type="OpenAIResponsesLLMService.Settings" required>
  Request settings for the backend model, the same object
  [`OpenAIResponsesLLMService`](/api-reference/server/services/llm/openai-responses)
  takes. `model` is required. `system_instruction` becomes the backend's
  instructions, `max_completion_tokens` its `max_output_tokens`, and `reasoning`
  is sent when configured. Fields the GPT-Live API does not accept for a
  delegated model (`temperature`, `top_p`, the penalties, `seed`, `top_k`,
  `max_tokens`) are dropped with a warning.
</ParamField>

<ParamField path="service_tier" type="str | None" default="None">
  Responses API service tier for delegated requests: `"auto"`, `"default"`,
  `"flex"` or `"priority"`.
</ParamField>

The backend model's tools and `tool_choice` come from the pipeline's `LLMContext`. Its function calls are executed by the service with the handlers registered for those tools, and results go back to the API as soon as they are available. Tools changed with `LLMSetToolsFrame` reach the session as a sparse update.

### ClientDelegation

A Pipecat worker is the backend. Passed as `delegation=OpenAILiveLLMService.ClientDelegation(...)`.

<ParamField path="backend" type="BaseWorker" required>
  The worker that runs delegated tasks, normally a
  [`BackendLLMWorker`](#backendllmworker) wrapping any LLM service. The service
  registers it as a child of the pipeline worker at setup, so the pipeline must
  run under a `WorkerRunner`.
</ParamField>

<ParamField path="timeout_secs" type="float" default="120.0">
  How long a delegation may take before it is abandoned. The model is told that
  the delegated work could not be completed.
</ParamField>

A delegation names no task: the GPT-Live model signals only that it is handing work over. The service sends the backend the transcript of the conversation since the previous delegation, and the backend works out the request from it. What comes back is appended to the live session according to each output's `prefers_spoken` flag: text the backend wants heard is appended as commentary, which the model relays in its own words, and the rest as silent context the model can draw on.

### BackendLLMWorker

`pipecat.workers.llm.BackendLLMWorker` runs an LLM service as the backend a GPT-Live model delegates to. It owns the backend's conversation, an `LLMContext` plus aggregator pair, so multi-step tool calling works as it does in any pipeline.

<ParamField path="llm" type="LLMService" required>
  The backend LLM service.
</ParamField>

<ParamField path="context" type="LLMContext" default="None">
  The backend's context, typically carrying its tools. A fresh empty context
  when omitted.
</ParamField>

<ParamField path="name" type="str" default="None">
  Worker name; auto-generated when omitted.
</ParamField>

<ParamField path="transform_output" type="Callable[[BackendOutput], Awaitable[BackendOutput]]" default="None">
  Called with each [`BackendOutput`](#backendoutput) before it is sent, to
  adjust its text or whether the user may hear it. Without one, only the final
  answer asks to be spoken.
</ParamField>

<ParamField path="user_params" type="LLMUserAggregatorParams" default="None">
  Parameters for the backend's user aggregator. Defaults to
  `ExternalUserTurnStrategies`, since the backend has no audio.
</ParamField>

<ParamField path="assistant_params" type="LLMAssistantAggregatorParams" default="None">
  Parameters for the backend's assistant aggregator.
</ParamField>

### BackendOutput

One piece of output from a backend, on its way to the GPT-Live model.

| Parameter        | Type   | Default | Description                                                                                                                                                       |
| ---------------- | ------ | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `text`           | `str`  |         | The text the backend produced.                                                                                                                                    |
| `is_thought`     | `bool` | `False` | Whether this is a reasoning summary rather than a response.                                                                                                       |
| `is_final`       | `bool` | `False` | Whether this is the backend's answer to the delegation, as opposed to progress on the way to it.                                                                  |
| `prefers_spoken` | `bool` | `True`  | Whether the backend would like the user to hear this. A hint: the GPT-Live model takes it into consideration but decides what to speak based on the conversation. |

## Usage

### Responses Delegation Mode

```python theme={null}
import os
from pipecat.frames.frames import LLMRunFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.worker import PipelineWorker
from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.processors.aggregators.llm_response_universal import LLMContextAggregatorPair
from pipecat.services.llm_service import FunctionCallParams
from pipecat.services.openai.live.llm import OpenAILiveLLMService
from pipecat.services.openai.responses.llm import OpenAIResponsesLLMService
from pipecat.workers.runner import WorkerRunner


async def get_current_weather(params: FunctionCallParams, location: str, format: str):
    """Get the current weather.

    Args:
        location: The city and state, e.g. "San Francisco, CA".
        format: The temperature unit, "celsius" or "fahrenheit".
    """
    await params.result_callback({"conditions": "nice", "temperature": 75, "format": format})


llm = OpenAILiveLLMService(
    api_key=os.getenv("OPENAI_API_KEY"),
    settings=OpenAILiveLLMService.Settings(
        system_instruction="You are a friendly, concise voice assistant. Delegate when the user asks for current information.",
    ),
    delegation=OpenAILiveLLMService.ResponsesDelegation(
        settings=OpenAIResponsesLLMService.Settings(
            model="gpt-5.6-terra",
            system_instruction="Use the available tools to answer questions about the weather.",
            reasoning=OpenAIResponsesLLMService.ReasoningConfig(effort="low"),
        ),
    ),
)

# The context's tools are the backend model's tools; their handlers run here.
# The trailing developer message asks the model to open the conversation.
context = LLMContext(
    [{"role": "developer", "content": "Greet the user and ask how you can help."}],
    [get_current_weather],
)
user_aggregator, assistant_aggregator = LLMContextAggregatorPair(context)

pipeline = Pipeline(
    [
        transport.input(),
        user_aggregator,
        llm,
        transport.output(),
        assistant_aggregator,
    ]
)

worker = PipelineWorker(pipeline)
runner = WorkerRunner()
await runner.add_workers(worker)


@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
    # Start the GPT-Live session from the context.
    await worker.queue_frames([LLMRunFrame()])


await runner.run()
```

### Client Delegation Mode

Any Pipecat LLM service can be the backend, with its own context and tools:

```python theme={null}
from pipecat.services.anthropic.llm import AnthropicLLMService
from pipecat.workers.llm import BackendLLMWorker

backend = BackendLLMWorker(
    llm=AnthropicLLMService(
        api_key=os.getenv("ANTHROPIC_API_KEY"),
        settings=AnthropicLLMService.Settings(
            system_instruction="You are the backend of a voice assistant. Each message is the recent conversation as a transcript; work out what is being asked and answer it.",
        ),
    ),
    context=LLMContext(tools=[get_current_weather]),
)

llm = OpenAILiveLLMService(
    api_key=os.getenv("OPENAI_API_KEY"),
    settings=OpenAILiveLLMService.Settings(system_instruction=FRONTEND_INSTRUCTIONS),
    delegation=OpenAILiveLLMService.ClientDelegation(backend=backend),
)

# The GPT-Live model's own context: no tools here, they belong to the backend.
context = LLMContext(
    [{"role": "developer", "content": "Greet the user and ask how you can help."}],
)
```

The service adds `backend` to the runner as a child of the pipeline worker, so the pipeline must run under a `WorkerRunner` as in the example above.

### Choosing What the User Hears

By default only the backend's final answer asks to be spoken. A `transform_output` callback sees every `BackendOutput` before it leaves the worker and can change that, or rewrite the text:

```python theme={null}
from pipecat.workers.llm import BackendLLMWorker, BackendOutput


async def transform_output(output: BackendOutput) -> BackendOutput:
    # Let the user hear progress the backend marks with a leading ">>".
    if output.text.startswith(">>"):
        return BackendOutput(
            text=output.text[2:].strip(),
            is_thought=output.is_thought,
            is_final=output.is_final,
            prefers_spoken=True,
        )
    return output


backend = BackendLLMWorker(llm=..., context=..., transform_output=transform_output)
```

### Restoring a Conversation

The GPT-Live API takes conversation history only at session start. To replace the context, for example with a saved conversation, set the messages and start a new session:

```python theme={null}
async def load_conversation(params: FunctionCallParams, filename: str):
    with open(filename) as file:
        params.context.set_messages(json.load(file))
    params.context.add_message(
        {
            "role": "developer",
            "content": "The saved conversation above has just been restored. Briefly tell the user it's loaded.",
        }
    )
    await params.result_callback({"status": "loaded"})
    await params.llm.reset_conversation()
```

## Notes

* **Turn taking is the model's**: The service proposes user turn boundaries from the API's transcript stream, and `LLMContextAggregatorPair` resolves them with the recommended `ExternalUserTurnStrategies(enable_interruptions=False)`. No `InterruptionFrame` is broadcast: the model handles being talked over itself, and a delegation keeps running when that happens. As a consequence every tool behaves as `cancel_on_interruption=False`; use `cancellable_by_llm=True` for tools the model should be able to cancel on request.
* **No local VAD**: The model detects the user's turns itself, so the pipeline needs no VAD analyzer and no `realtime_service_mode` on the aggregators. The user aggregator writes each user turn to the context as it ends, before any tool calls it triggers.
* **Session start**: The session starts on the first `LLMContextFrame`, typically queued as an `LLMRunFrame`. The context's leading system message (or `Settings.system_instruction`) becomes the model's instructions and the remaining text messages seed the session as prior conversation. At most 128 messages are kept, dropping the oldest; tool calls, tool results and non-text content have no representation in the startup history and are skipped.
* **Opening the conversation**: A trailing developer message in the context is delivered to the model once the session starts, as the request to speak first. Omit it to have the model wait for the user.
* **System instruction precedence**: `system_instruction` from service settings takes precedence over an initial system message in the LLM context. A warning is logged when both are set.
* **Tools belong to the backend**: The GPT-Live model calls no tools of its own. In Responses delegation mode this is invisible to your code: register functions and write their handlers as for any other LLM service, and the backend model calls them through the pipeline. In client delegation mode the `BackendLLMWorker`'s own context carries the tools, and its pipeline runs them.
* **Audio format**: Input audio is resampled to 24 kHz; output is 24 kHz PCM, pushed as `SpeechOutputAudioRawFrame` so the output transport derives bot speaking state from the audio itself.
* **Usage metrics**: GPT-Live usage is reported as cumulative session seconds. Token usage comes from the backend model's completed responses in Responses delegation.
* **Delegation failures**: When a client delegation fails or times out, the model is told the delegated work could not be completed and the detail is pushed as an `ErrorFrame`. A delegation that finishes without producing any text is reported to the model as having no answer, so the conversation moves on.

## Event Handlers

| Event                   | Description                                                          |
| ----------------------- | -------------------------------------------------------------------- |
| `on_session_started`    | Called with the session resource once the session is ready for audio |
| `on_delegation_created` | Called with the delegation metadata when the model delegates work    |

```python theme={null}
@llm.event_handler("on_session_started")
async def on_session_started(service, session):
    print(f"Session started: {session.id}")

@llm.event_handler("on_delegation_created")
async def on_delegation_created(service, delegation):
    print(f"Delegated to the {delegation.target} backend: {delegation.id}")
```