OpenAI LLM

Overview

OpenAILLMService provides chat completion capabilities using OpenAI’s API, supporting streaming responses, function calling, vision input, and advanced context management for conversational AI applications with state-of-the-art language models.

OpenAI LLM API Reference

Pipecat’s API methods for OpenAI integration

Example Implementation

Function calling example with weather API

OpenAI Documentation

Official OpenAI API documentation

OpenAI Platform

Access models and manage API keys

Installation

To use OpenAI services, install the required dependencies:

uv add "pipecat-ai[openai]"

Prerequisites

OpenAI Account Setup

Before using OpenAI LLM services, you need:

OpenAI Account: Sign up at OpenAI Platform
API Key: Generate an API key from your account dashboard
Model Selection: Choose from available models (GPT-4.1, GPT-4o, GPT-4o-mini, etc.)
Usage Limits: Set up billing and usage limits as needed

Required Environment Variables

OPENAI_API_KEY: Your OpenAI API key for authentication

Configuration

str

default:"None"

deprecated

Deprecated since 0.0.105, removed in 2.0.0. Use settings=OpenAILLMService.Settings(...) instead.

str

default:"None"

OpenAI API key. If None, uses the OPENAI_API_KEY environment variable.

str

default:"None"

Custom base URL for the OpenAI API. Override for proxied or self-hosted deployments.

str

default:"None"

OpenAI organization ID.

str

default:"None"

OpenAI project ID.

Mapping[str, str]

default:"None"

Additional HTTP headers to include in every request.

str | None

default:"None"

Service tier to request, such as "auto", "flex", or "priority". When None, the parameter is omitted and OpenAI’s default tier applies.

OpenAILLMService.Settings

default:"None"

Runtime-configurable model settings. See Settings below.

InputParams

default:"None"

deprecated

Deprecated since 0.0.105, removed in 2.0.0. Use settings=OpenAILLMService.Settings(...) instead.

float

default:"5.0"

Request timeout in seconds. Used when retry_on_timeout is enabled to determine when to retry.

bool

default:"False"

Whether to retry the request once if it times out. The retry attempt has no timeout limit.

Settings

Runtime-configurable settings passed via the settings constructor argument using OpenAILLMService.Settings(...). These can be updated mid-conversation with LLMUpdateSettingsFrame. See Service Settings for details.

Parameter	Type	Default	Description
`model`	`str`	`"gpt-4.1"`	OpenAI model identifier. (Inherited from base settings.)
`system_instruction`	`str`	`None`	System instruction/prompt for the model. (Inherited from base settings.)
`temperature`	`float`	`NOT_GIVEN`	Sampling temperature (0.0 to 2.0). Lower values are more focused, higher values are more creative.
`max_tokens`	`int`	`NOT_GIVEN`	Maximum tokens to generate.
`top_p`	`float`	`NOT_GIVEN`	Top-p (nucleus) sampling (0.0 to 1.0). Controls diversity of output.
`top_k`	`int`	`None`	Top-k sampling parameter. Not sent by this service, so it has no effect.
`frequency_penalty`	`float`	`NOT_GIVEN`	Penalty for frequent tokens (-2.0 to 2.0). Positive values discourage repetition.
`presence_penalty`	`float`	`NOT_GIVEN`	Penalty for new topics (-2.0 to 2.0). Positive values encourage the model to talk about new topics.
`seed`	`int`	`NOT_GIVEN`	Random seed for deterministic outputs.
`max_completion_tokens`	`int`	`NOT_GIVEN`	Maximum completion tokens to generate.

NOT_GIVEN values are omitted from the API request entirely, letting the OpenAI API use its own defaults. This is different from None, which would be sent explicitly.

Usage

Basic Setup

import os

from pipecat.services.openai.llm import OpenAILLMService

llm = OpenAILLMService(
    api_key=os.getenv("OPENAI_API_KEY"),
    settings=OpenAILLMService.Settings(model="gpt-4o"),
)

With Custom Settings

from pipecat.services.openai.llm import OpenAILLMService

llm = OpenAILLMService(
    api_key=os.getenv("OPENAI_API_KEY"),
    settings=OpenAILLMService.Settings(
        model="gpt-4.1",
        temperature=0.7,
        max_completion_tokens=1000,
        frequency_penalty=0.5,
    ),
)

Updating Settings at Runtime

Model settings can be changed mid-conversation using LLMUpdateSettingsFrame:

from pipecat.frames.frames import LLMUpdateSettingsFrame
from pipecat.services.openai.base_llm import OpenAILLMSettings

await worker.queue_frame(
    LLMUpdateSettingsFrame(
        delta=OpenAILLMSettings(
            temperature=0.3,
            max_completion_tokens=500,
        )
    )
)

Notes

OpenAI-compatible providers: Many third-party LLM providers offer OpenAI-compatible APIs. You can use OpenAILLMService with them by setting base_url to the provider’s endpoint.
Retry behavior: When retry_on_timeout=True, the first attempt uses the retry_timeout_secs timeout. If it times out, a second attempt is made with no timeout limit.
Function calling: Supports OpenAI’s tool/function calling format. Register function handlers with llm.register_function() to handle tool calls automatically.
System instruction precedence: If both the system_instruction setting and an initial system message in the context are set, both are sent, with system_instruction first, and a warning is logged.

Event Handlers

OpenAILLMService supports the following event handlers, inherited from LLMService:

Event	Description
`on_completion_timeout`	Called when an LLM completion request times out
`on_function_calls_started`	Called when function calls are received and execution is about to start
`on_function_calls_cancelled`	Called after one or more in-progress function calls are cancelled

@llm.event_handler("on_completion_timeout")
async def on_completion_timeout(service):
    print("LLM completion timed out")

Get Started

Pipecat Server

Client SDKs

Pipecat Flows

Pipecat Cloud

CLI

Pipecat Context Hub

Overview

OpenAI LLM API Reference

Example Implementation

OpenAI Documentation

OpenAI Platform

Installation

Prerequisites

OpenAI Account Setup

Required Environment Variables

Configuration

Settings

Usage

Basic Setup

With Custom Settings

Updating Settings at Runtime

Notes

Event Handlers

​Overview

OpenAI LLM API Reference

Example Implementation

OpenAI Documentation

OpenAI Platform

​Installation

​Prerequisites

​OpenAI Account Setup

​Required Environment Variables

​Configuration

​Settings

​Usage

​Basic Setup

​With Custom Settings

​Updating Settings at Runtime

​Notes

​Event Handlers

Overview

Installation

Prerequisites

OpenAI Account Setup

Required Environment Variables

Configuration

Settings

Usage

Basic Setup

With Custom Settings

Updating Settings at Runtime

Notes

Event Handlers