Skip to main content

Things you’ll need

  • An active Twilio account with API credentials
  • One or more Twilio provisioned phone numbers
  • A tunneling service like ngrok for local development
  • API keys for speech-to-text, text-to-speech, and LLM services

Phone Number Setup

You’ll need Twilio phone numbers for both dial-in and dial-out functionality:
  • Visit console.twilio.com and purchase phone numbers
  • Ensure your numbers support Voice capabilities
  • Configure webhook URLs for dial-in numbers (covered below)

Environment Setup

Configure your environment variables for Twilio and AI services:
.env
TWILIO_ACCOUNT_SID=...
TWILIO_AUTH_TOKEN=...
OPENAI_API_KEY=...
DEEPGRAM_API_KEY=...
CARTESIA_API_KEY=...

Dial-in

Dial-in allows users to call your Twilio number and connect to your Pipecat bot via WebSocket Media Streams.

How It Works

Here’s the sequence of events when someone calls your Twilio number:
  1. Twilio sends WebSocket messages: Twilio processes the associated TwiML Bin and starts a WebSocket stream to your bot (local or Pipecat Cloud)
  2. Parse the WebSocket messages: Your bot parses the WebSocket connection messages to set up the corresponding Pipecat transport
  3. (Optional) Look up the caller: Optionally, look up the caller using Twilio’s REST API to retrieve custom information about the call and personalize your bot’s behavior
  4. Bot starts responding: Once the pipeline is started, your bot will initiate the conversation

Setting Up Twilio

1. Create a TwiML Bin

A TwiML Bin tells Twilio how to handle incoming calls. You’ll create one that establishes a WebSocket connection to your bot.
  1. Go to the Twilio Console
  2. Navigate to TwiML BinsMy TwiML Bins
  3. Click the + to create a new TwiML Bin
  4. Name your bin and add the TwiML:
  • Local Development
  • Pipecat Cloud
<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Connect>
    <Stream url="wss://your-url.ngrok.io/ws" />
  </Connect>
</Response>
Replace your-url.ngrok.io with your ngrok URL.
  1. Click Save

2. Assign TwiML Bin to Your Phone Number

  1. Navigate to Phone NumbersManageActive Numbers
  2. Click on your Twilio phone number
  3. In the “Voice Configuration” section:
    • Set “A call comes in” to TwiML Bin
    • Select the TwiML Bin you created
  4. Click Save configuration

Running the Example

You’ll need two terminal windows to run the example.
  1. In your first terminal, start ngrok to expose your local server:
    ngrok http 7860
    
    Tip: Use the --subdomain flag for a reusable ngrok URL.
  2. In your second terminal, install dependencies:
    uv sync
    
  3. Then run the bot:
    uv run bot.py -t twilio
    
  4. Call your bot by placing a call to the number associated with your bot. The bot will answer and start the conversation.

Personalizing Your Bot with Caller Information

When a call comes in, your bot receives a Call SID from Twilio. You can use this Call SID to fetch caller information from Twilio’s REST API, including the caller’s phone number. With this information, you can:
  • Look up customer information in your database
  • Personalize greetings and responses based on caller identity
  • Route calls to different bot behaviors (e.g., VIP handling, support vs sales)
  • Implement custom business logic based on the caller
The example bot demonstrates how to fetch caller information asynchronously using aiohttp to avoid blocking the event loop, ensuring your bot remains responsive even with multiple concurrent calls.

Complete Dial-in Example

See the full implementation with WebSocket parsing, REST API integration, and caller personalization

Dial-out

Dial-out allows your bot to initiate calls to phone numbers using Twilio’s outbound calling capabilities with WebSocket Media Streams.

How It Works

Here’s the sequence of events for dial-out calls:
  1. Your application triggers a dial-out (via API call or user action)
  2. Server initiates a Twilio call to the target phone number
  3. Twilio establishes the call and opens a WebSocket connection
  4. Your bot joins the WebSocket and sets up the pipeline
  5. The recipient answers and is connected to your bot
  6. The bot handles the conversation with real-time audio streaming

Set up your server for dial-out

The dial-out server creates outbound calls and manages WebSocket connections. When you trigger a dial-out call, your server:
  1. Receives the dial-out request with target phone number and parameters
  2. Creates a Twilio call using the REST API with TwiML that establishes WebSocket
  3. Accepts the WebSocket connection from Twilio
  4. Parses call data including any custom parameters
  5. Runs the Pipecat bot with the call information

Complete Server Implementation

See the full FastAPI server code with outbound call creation and WebSocket handling

Configure your Pipecat bot for dial-out

The dial-out bot configuration is similar to dial-in, but you can pass custom parameters when creating the outbound call: Custom Parameters for Dial-out: Pass information to your bot through TwiML parameters:
server.py
# When creating the outbound call, include custom parameters
twiml = f"""
<Response>
  <Connect>
    <Stream url="wss://your-server.com/ws">
      <Parameter name="campaign_id" value="summer_sale" />
      <Parameter name="customer_id" value="{customer_id}" />
      <Parameter name="call_type" value="outbound" />
    </Stream>
  </Connect>
</Response>
"""

# Create the outbound call
call = twilio_client.calls.create(
    to=target_number,
    from_=your_twilio_number,
    twiml=twiml
)
Bot Configuration: The bot receives and uses these parameters:
bot.py
# Parse WebSocket data (same as dial-in)
transport_type, call_data = await parse_telephony_websocket(websocket)

# Access custom parameters
custom_params = call_data["body"]
campaign_id = custom_params.get("campaign_id")
customer_id = custom_params.get("customer_id")

# Customize bot behavior for outbound calls
if campaign_id == "summer_sale":
    greeting = f"Hi! I'm calling about our summer sale promotion..."

Complete Bot Implementation

See the full bot.py with outbound call handling and parameter usage

Run the Example

To test dial-out functionality:
  1. Start your server: Run your FastAPI server
  2. Trigger a call: Make an API request to start an outbound call
  3. Answer your phone: The bot will call the specified number
  4. Talk to your bot: Have a conversation with your AI agent

Complete Setup Instructions

See the full README with step-by-step setup, API usage, and outbound call configuration

Key Features

Audio Format and Sample Rate

Twilio Media Streams uses 8kHz mono audio with 16-bit PCM encoding. To avoid resampling, set the audio input and output sample rate to 8000 Hz:
task = PipelineTask(
    pipeline,
    params=PipelineParams(
        audio_in_sample_rate=8000,
        audio_out_sample_rate=8000,
        allow_interruptions=True,
    ),
)

Automatic Call Termination

When you provide Twilio credentials to the TwilioFrameSerializer, it automatically ends calls when your pipeline ends:
serializer = TwilioFrameSerializer(
    stream_sid=stream_id,
    call_sid=call_id,
    account_sid=os.getenv("TWILIO_ACCOUNT_SID"),
    auth_token=os.getenv("TWILIO_AUTH_TOKEN"),
)

Deployment

Pipecat Cloud Deployment

For production deployment without managing your own infrastructure, use Pipecat Cloud:

Twilio WebSocket on Pipecat Cloud

Deploy Twilio WebSocket bots with automatic scaling and managed infrastructure

Self-Hosted Production Deployment

For fully self-hosted production deployment, ensure your servers are:
  • Publicly accessible with HTTPS
  • Able to handle concurrent WebSocket connections
  • Properly configured with Twilio API credentials
  • Implementing proper error handling and logging

Next Steps