Twilio WebSocket Integration

Things you’ll need

An active Twilio account with API credentials
One or more Twilio provisioned phone numbers
A public-facing server or tunneling service like ngrok
API keys for speech-to-text, text-to-speech, and LLM services

Twilio Dial-in Example

Complete dial-in implementation using Twilio Media Streams over WebSocket

Twilio Dial-out Example

Outbound calling using Twilio Media Streams with WebSocket transport

Phone Number Setup

You’ll need Twilio phone numbers for both dial-in and dial-out functionality:

Visit console.twilio.com and purchase phone numbers
Ensure your numbers support Voice capabilities
Configure webhook URLs for dial-in numbers (covered below)

Environment Setup

Configure your environment variables for Twilio and AI services:

.env

TWILIO_ACCOUNT_SID=...
TWILIO_AUTH_TOKEN=...
OPENAI_API_KEY=...
DEEPGRAM_API_KEY=...
CARTESIA_API_KEY=...

Dial-in

Dial-in allows users to call your Twilio number and connect to your Pipecat bot via WebSocket Media Streams.

How It Works

Here’s the sequence of events when someone calls your Twilio number:

Twilio receives an incoming call to your phone number
Twilio calls your webhook server with call details
Your server responds with TwiML that establishes a WebSocket connection
Twilio opens a WebSocket to your server with real-time audio
Your bot processes the audio using the Pipecat pipeline
The bot responds with audio sent back to Twilio over WebSocket
Twilio plays the audio to the caller in real-time

Set up your webhook server

Your server handles Twilio webhooks and WebSocket connections for real-time audio streaming. When Twilio receives a call, your server:

Receives the webhook with call details (CallSid, caller information)
Returns TwiML that establishes a WebSocket connection to your server
Accepts the WebSocket connection from Twilio
Parses call data including custom parameters from the WebSocket
Runs the Pipecat bot with the parsed call information

The server orchestrates the real-time audio streaming between Twilio’s telephony infrastructure and your Pipecat bot pipeline.

Custom Parameters with TwiML

Twilio’s <Parameter> noun allows you to pass custom information to your bot through the WebSocket connection. This is essential for passing caller details, user IDs, or other contextual information:

TwiML with custom parameters

<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Connect>
    <Stream url="wss://your-server.com/ws">
      <Parameter name="caller_id" value="+1234567890" />
      <Parameter name="user_id" value="user_123" />
      <Parameter name="session_type" value="support" />
    </Stream>
  </Connect>
  <Pause length="40"/>
</Response>

These parameters are accessible in your bot through Pipecat’s built-in WebSocket parser, which extracts them from the customParameters field in the WebSocket data.

Complete Server Implementation

See the full FastAPI server code with TwiML generation and WebSocket handling

Configure your Pipecat bot for dial-in

Your bot receives the WebSocket connection and parses the call data to extract custom parameters and call information. The key components are: WebSocket Parsing: Pipecat’s built-in parser extracts call data from Twilio’s WebSocket messages:

bot.py

from pipecat.runner.utils import parse_telephony_websocket

async def run_bot(websocket: WebSocket):
    # Parse Twilio WebSocket data
    transport_type, call_data = await parse_telephony_websocket(websocket)

    # Extract call information
    stream_id = call_data["stream_id"]
    call_id = call_data["call_id"]

    # Access custom parameters from TwiML
    custom_params = call_data["body"]  # Contains all <Parameter> values
    caller_id = custom_params.get("caller_id")
    user_id = custom_params.get("user_id")
    session_type = custom_params.get("session_type")

bot.py

# Create Twilio serializer with call details
serializer = TwilioFrameSerializer(
    stream_sid=stream_id,
    call_sid=call_id,
    account_sid=os.getenv("TWILIO_ACCOUNT_SID"),
    auth_token=os.getenv("TWILIO_AUTH_TOKEN"),
)

# Configure WebSocket transport
transport = FastAPIWebsocketTransport(
    websocket=websocket,
    params=FastAPIWebsocketParams(
        audio_in_enabled=True,
        audio_out_enabled=True,
        add_wav_header=False,
        vad_analyzer=SileroVADAnalyzer(),
        serializer=serializer,
    ),
)

# Your bot pipeline setup here...

Custom Parameter Usage: Use the extracted parameters to customize your bot’s behavior:

bot.py

# Customize bot behavior based on parameters
if session_type == "support":
    system_prompt = "You are a helpful customer support agent..."
elif session_type == "sales":
    system_prompt = "You are a friendly sales representative..."
else:
    system_prompt = "You are a general assistant..."

# Use caller_id for personalization
greeting = f"Hello! I see you're calling from {caller_id}. How can I help you today?"

Complete Bot Implementation

See the full bot.py with WebSocket parsing, transport setup, and pipeline configuration

Set up Twilio webhook

Configure your Twilio phone number to use your server’s webhook URL:

Go to the Twilio Console
Navigate to Phone Numbers → Manage → Active Numbers
Click on your phone number
Under “Configure”, set “A Call Comes In” to:
- Webhook: https://your-server.com/ (your server’s URL)
- HTTP Method: POST

Run the Example

For local development, use ngrok to expose your local server:

# Start your server
python server.py

# In another terminal, start ngrok
ngrok http 8000

# Use the ngrok URL as your Twilio webhook

Complete Setup Instructions

See the full README with step-by-step setup, webhook configuration, and testing

Dial-out

Dial-out allows your bot to initiate calls to phone numbers using Twilio’s outbound calling capabilities with WebSocket Media Streams.

How It Works

Here’s the sequence of events for dial-out calls:

Your application triggers a dial-out (via API call or user action)
Server initiates a Twilio call to the target phone number
Twilio establishes the call and opens a WebSocket connection
Your bot joins the WebSocket and sets up the pipeline
The recipient answers and is connected to your bot
The bot handles the conversation with real-time audio streaming

Set up your server for dial-out

The dial-out server creates outbound calls and manages WebSocket connections. When you trigger a dial-out call, your server:

Receives the dial-out request with target phone number and parameters
Creates a Twilio call using the REST API with TwiML that establishes WebSocket
Accepts the WebSocket connection from Twilio
Parses call data including any custom parameters
Runs the Pipecat bot with the call information

Complete Server Implementation

See the full FastAPI server code with outbound call creation and WebSocket handling

Configure your Pipecat bot for dial-out

The dial-out bot configuration is similar to dial-in, but you can pass custom parameters when creating the outbound call: Custom Parameters for Dial-out: Pass information to your bot through TwiML parameters:

server.py

# When creating the outbound call, include custom parameters
twiml = f"""
<Response>
  <Connect>
    <Stream url="wss://your-server.com/ws">
      <Parameter name="campaign_id" value="summer_sale" />
      <Parameter name="customer_id" value="{customer_id}" />
      <Parameter name="call_type" value="outbound" />
    </Stream>
  </Connect>
</Response>
"""

# Create the outbound call
call = twilio_client.calls.create(
    to=target_number,
    from_=your_twilio_number,
    twiml=twiml
)

Bot Configuration: The bot receives and uses these parameters:

bot.py

# Parse WebSocket data (same as dial-in)
transport_type, call_data = await parse_telephony_websocket(websocket)

# Access custom parameters
custom_params = call_data["body"]
campaign_id = custom_params.get("campaign_id")
customer_id = custom_params.get("customer_id")

# Customize bot behavior for outbound calls
if campaign_id == "summer_sale":
    greeting = f"Hi! I'm calling about our summer sale promotion..."

Complete Bot Implementation

See the full bot.py with outbound call handling and parameter usage

Run the Example

To test dial-out functionality:

Start your server: Run your FastAPI server
Trigger a call: Make an API request to start an outbound call
Answer your phone: The bot will call the specified number
Talk to your bot: Have a conversation with your AI agent

Complete Setup Instructions

See the full README with step-by-step setup, API usage, and outbound call configuration

Key Features

Audio Format and Sample Rate

Twilio Media Streams uses 8kHz mono audio with 16-bit PCM encoding. To avoid resampling, set the audio input and outputsample rate to 8000 Hz:

task = PipelineTask(
    pipeline,
    params=PipelineParams(
        audio_in_sample_rate=8000,
        audio_out_sample_rate=8000,
        allow_interruptions=True,
    ),
)

Automatic Call Termination

When you provide Twilio credentials to the TwilioFrameSerializer, it automatically ends calls when your pipeline ends:

serializer = TwilioFrameSerializer(
    stream_sid=stream_id,
    call_sid=call_id,
    account_sid=os.getenv("TWILIO_ACCOUNT_SID"),
    auth_token=os.getenv("TWILIO_AUTH_TOKEN"),
)

Deployment

Local Development

Use ngrok to expose your local server for testing:

python server.py
ngrok http 8000
# Use the ngrok URL as your Twilio webhook

Pipecat Cloud Deployment

For production deployment without managing your own infrastructure, use Pipecat Cloud:

Twilio WebSocket on Pipecat Cloud

Deploy Twilio WebSocket bots with automatic scaling and managed infrastructure

Self-Hosted Production Deployment

For fully self-hosted production deployment, ensure your servers are:

Publicly accessible with HTTPS
Able to handle concurrent WebSocket connections
Properly configured with Twilio API credentials
Implementing proper error handling and logging

Next Steps

Explore the complete examples for full implementations
Learn about Daily + Twilio SIP integration for advanced telephony scenarios
Check out Daily PSTN integration for direct phone number provisioning

Learning Pipecat

Fundamentals

Features

Telephony

Twilio WebSocket Integration

Things you’ll need

Twilio Dial-in Example

Twilio Dial-out Example

Phone Number Setup

Environment Setup

Dial-in

How It Works

Set up your webhook server

Custom Parameters with TwiML

Complete Server Implementation

Configure your Pipecat bot for dial-in

Complete Bot Implementation

Set up Twilio webhook

Run the Example

Complete Setup Instructions

Dial-out

How It Works

Set up your server for dial-out

Complete Server Implementation

Configure your Pipecat bot for dial-out

Complete Bot Implementation

Run the Example

Complete Setup Instructions

Key Features

Audio Format and Sample Rate

Automatic Call Termination

Deployment

Local Development

Pipecat Cloud Deployment

Twilio WebSocket on Pipecat Cloud

Self-Hosted Production Deployment

Next Steps

Learning Pipecat

Fundamentals

Features

Telephony

​Things you’ll need

Twilio Dial-in Example

Twilio Dial-out Example

​Phone Number Setup

​Environment Setup

​Dial-in

​How It Works

​Set up your webhook server

​Custom Parameters with TwiML

Complete Server Implementation

​Configure your Pipecat bot for dial-in

Complete Bot Implementation

​Set up Twilio webhook

​Run the Example

Complete Setup Instructions

​Dial-out

​How It Works

​Set up your server for dial-out

Complete Server Implementation

​Configure your Pipecat bot for dial-out

Complete Bot Implementation

​Run the Example

Complete Setup Instructions

​Key Features

​Audio Format and Sample Rate

​Automatic Call Termination

​Deployment

​Local Development

​Pipecat Cloud Deployment

Twilio WebSocket on Pipecat Cloud

​Self-Hosted Production Deployment

​Next Steps

Things you’ll need

Phone Number Setup

Environment Setup

Dial-in

How It Works

Set up your webhook server

Custom Parameters with TwiML

Configure your Pipecat bot for dial-in

Set up Twilio webhook

Run the Example

Dial-out

How It Works

Set up your server for dial-out

Configure your Pipecat bot for dial-out

Run the Example

Key Features

Audio Format and Sample Rate

Automatic Call Termination

Deployment

Local Development

Pipecat Cloud Deployment

Self-Hosted Production Deployment

Next Steps