Dialout: WebRTC (Daily)

Pipecat

Pipecat is a framework for building voice-enabled, real-time, multimodal AI applications

Overview

Get Pipecat and its required services installed on your machine

Installation & Setup

Build and run your first Pipecat application

Quickstart

Understanding how Pipecat works: frames, processors, and pipelines.

Core Concepts

Where to go next with Pipecat based on your project needs

Next Steps & Examples

Learn how to deploy, scale, and extend your Pipecat applications

Guides

Learn how to detect and respond when users are inactive in conversations

Detecting Idle Users

Best practices for properly terminating Pipecat pipelines

Ending a Pipeline

Enable LLMs to interact with external services and APIs

Function Calling

Learn how to control when user speech is processed in your conversational bot

Muting User Input

User Input Muting with STTMuteFilter

Learn how to record and save audio from conversations between users and your bot

Recording Audio

Recording Conversation Audio

Learn how to collect and save conversation transcripts between users and your bot

Recording Transcripts

Recording Conversation Transcripts

Create a real-time AI chatbot using Gemini Multimodal Live and Pipecat

Gemini Multimodal Live

Building with Gemini Multimodal Live

Learn how to monitor performance and LLM/TTS usage with Pipecat.

Metrics

Learn how to integrate Krisp noise cancellation into your Pipecat application

Noise cancellation with Krisp

Create voice agents with OpenAI audio models and Pipecat

OpenAI Audio Models and APIs

Building With OpenAI Audio Models and APIs

Learn how to create structured conversations using Pipecat’s flow system

Pipecat Flows

How to call your bots (or have them call you)

Dial-in: WebRTC (Daily)

Connect phone calls to your Pipecat bot using Twilio and Daily's SIP integration

Dial-in: WebRTC (Twilio + Daily)

Call your Pipecat bot over websockets using Twilio

Dial-in: Twilio (Media Streams)

Basic deployment pattern for Pipecat bots

Deployment pattern

Deploy Pipecat agents with managed infrastructure

Example: Pipecat Cloud

Example: Fly.io

Example: Cerebrium

Example: Modal

Complete reference for Pipecat’s Python server APIs and services

Server API Reference

Reference docs

AI services integrated with Pipecat and their setup requirements

Supported Services

Advanced conversational turn detection powered by the smart-turn model

Smart Turn Detection

Factory for creating and managing user and assistant transcript processors with shared event handling

TranscriptProcessor

A processor that monitors user inactivity and triggers callbacks after specified timeout periods

UserIdleProcessor

Technical reference for Pipecat’s conversation flow system

Configure pipeline execution with PipelineParams

PipelineParams

Manage pipeline execution and lifecycle with PipelineTask

PipelineTask

Automatically detect and handle idle pipelines with no bot activity

Pipeline Idle Detection

Monitor pipeline health with heartbeat frames

Pipeline Heartbeats

Run multiple pipeline branches in parallel, with synchronized inputs and outputs for complex flows

ParallelPipeline

Overview of Pipecat’s core service implementations

Base implementations for handling audio/video I/O streams

Transport

Base implementations for speech-to-text and text-to-speech services

Speech Service

Base implementation for Large Language Model services

LLM Service

Service to connect to MCP (Model Context Protocol) servers

MCPClient

Base implementations for image and vision services

Media Service

Client libraries for building real-time AI applications with Pipecat

Client SDKs

Build web applications with Pipecat’s JavaScript client library

SDK Introduction

Build React applications with Pipecat's React client library

Build React Native applications with Pipecat's React Native client library

API reference for the Pipecat React Native SDK

API Reference

Build iOS applications with Pipecat’s Swift client library

Build Android applications with Pipecat's Kotlin client library

Build native applications with Pipecat’s C++ client library

WebRTC implementation for C++ using Daily

Daily WebRTC Transport

Community

GitHub

Examples

Changelog

Getting Started

Server APIs

Join the Discord

WebRTC transport implementation using Daily for real-time audio/video communication

Daily WebRTC

A lightweight WebRTC transport for peer-to-peer audio and video communication in Pipecat

SmallWebRTCTransport

WebSocket transport implementation for FastAPI web applications

FastAPI WebSocket

WebSocket server transport implementation for real-time audio communication

WebSocket Server

Speech-to-text service implementation using AssemblyAI’s real-time transcription API

AssemblyAI

Speech-to-text service implementation using Amazon Transcribe’s real-time transcription API

AWS Transcribe

Speech-to-text service using Azure Cognitive Services Speech SDK

Azure

Speech-to-text service implementation using Deepgram’s real-time transcription API

Deepgram

Speech-to-text service implementation using Fal’s Wizper API

Fal (Wizper)

Speech-to-text service implementation using Gladia’s API

Gladia

Speech-to-text service implementation using Google Cloud’s Speech-to-Text V2 API

Google

Speech-to-text service implementation using Groq’s Whisper API

Groq (Whisper)

Speech-to-text service implementation using NVIDIA Riva

NVIDIA Riva

Speech-to-text service implementation using OpenAI’s Speech-to-Text APIs

OpenAI

Speech-to-text service implementation using a locally-loaded Ultravox multimodal model

Ultravox

Speech-to-text service implementation using locally-downloaded Whisper models

Whisper

Large Language Model service implementation using Anthropic’s Claude API

Anthropic

Large Language Model service implementation using Amazon Bedrock API

AWS Bedrock

Large Language Model service implementation using Azure OpenAI API

LLM service implementation using Cerebras’s API with OpenAI-compatible interface

Cerebras

LLM service implementation using DeepSeek’s API with OpenAI-compatible interface

DeepSeek

LLM service implementation using Fireworks AI’s API with OpenAI-compatible interface

Fireworks AI

Large Language Model service implementation using Google’s Gemini API

Google Gemini

LLM service implementation using Google’s Vertex AI with OpenAI-compatible interface

Google Vertex AI

LLM service implementation using Grok’s API with OpenAI-compatible interface

Grok

LLM service implementation using Groq’s API with OpenAI-compatible interface

Groq

LLM service implementation using NVIDIA’s NIM (NVIDIA Inference Microservice) API with OpenAI-compatible interface

NVIDIA NIM

LLM service implementation using Ollama with OpenAI-compatible interface

Ollama

Large Language Model services using OpenAI’s chat completion API

LLM service implementation using OpenPipe for LLM request logging and fine-tuning

OpenPipe

LLM service implementation using OpenRouter’s API with OpenAI-compatible interface

OpenRouter

LLM service implementation using Perplexity’s API with OpenAI-compatible interface

Perplexity

LLM service implementation using Alibaba Cloud’s Qwen models through an OpenAI-compatible interface

Qwen

LLM service implementation using Together AI’s API with OpenAI-compatible interface

Together AI

Text-to-speech service implementation using AWS Polly

AWS Polly

Text-to-speech service using Azure Cognitive Services Speech SDK

Text-to-speech services using Cartesia’s WebSocket and HTTP APIs

Cartesia

Text-to-speech service implementation using Deepgram’s Aura API

Text-to-speech service using ElevenLab’s streaming API with word-level timing

ElevenLabs

Real-time text-to-speech service using Fish Audio's WebSocket API

Fish Audio

Text-to-speech service using Google’s Cloud Text-to-Speech API

Text-to-speech service implementation using Groq’s TTS API

Text-to-speech service implementation using LMNT’s streaming API

LMNT

Text-to-speech service implementation using Neuphonic’s API

Neuphonic

Text-to-speech service implementation using NVIDIA Riva

Text-to-speech service using OpenAI’s TTS API

Text-to-speech service implementation using the Piper TTS server

Piper

Text-to-speech services using PlayHT’s WebSocket and HTTP APIs

PlayHT

Text-to-speech service implementations using Rime AI

Rime

Text-to-speech service implementation using Coqui’s XTTS streaming server

XTTS

Real-time speech-to-speech service implementation using AWS Nova Sonic

Fundamentals

Features

Telephony

Deploying your bot

Dialout: WebRTC (Daily)