Daily WebRTC Transport

Pipecat

Pipecat is a framework for building voice-enabled, real-time, multimodal AI applications

Overview

Get Pipecat and its required services installed on your machine

Installation & Setup

Build and run your first Pipecat application

Quickstart

Understanding how Pipecat works: frames, processors, and pipelines.

Core Concepts

Common applications you can build with Pipecat

Use Cases

Learn how to deploy, scale, and extend your Pipecat applications

Guides

Enable LLMs to interact with external services and APIs

Function Calling

Create a real-time AI chatbot using Gemini Multimodal Live and Pipecat

Gemini Multimodal Live

Building with Gemini Multimodal Live

Learn how to monitor performance and LLM/TTS usage with Pipecat.

Metrics

Learn how to integrate Krisp noise cancellation into your Pipecat application

Noise cancellation with Krisp

Learn how to create structured conversations using Pipecat’s flow system

Pipecat Flows

How to call your bots (or have them call you)

Dial-in: WebRTC (Daily)

Call your Pipecat bot using Daily WebRTC and a Twilio phone number

Dial-in: WebRTC (Twilio + Daily)

Call your Pipecat bot over websockets using Twilio

Dial-in: WebSockets (Twilio)

Dialout: WebRTC (Daily)

Basic deployment pattern for Pipecat bots

Deployment pattern

Deploy Pipecat agents with managed infrastructure

Example: Pipecat Cloud

Example: Fly.io

Example: Cerebrium

Complete reference for Pipecat’s Python server APIs and services

Server API Reference

Reference docs

AI services integrated with Pipecat and their setup requirements

Supported Services

Overview of Pipecat’s core service implementations

Base implementations for handling audio/video I/O streams

Transport

Base implementations for speech-to-text and text-to-speech services

Speech Service

Base implementation for Large Language Model services

LLM Service

Base implementations for text filtering and preprocessing

Text Processing

Base implementations for image and vision services

Media Service

Technical reference for Pipecat’s conversation flow system

Monitor pipeline health with heartbeat frames

Pipeline Heartbeats

Factory for creating and managing user and assistant transcript processors with shared event handling

TranscriptProcessor

A processor that monitors user inactivity and triggers callbacks after specified timeout periods

UserIdleProcessor

Client libraries for building real-time AI applications with Pipecat

Client SDKs

Build web applications with Pipecat’s JavaScript client library

SDK Introduction

Build React applications with Pipecat's React client library

Build React Native applications with Pipecat's React Native client library

API reference for the Pipecat React Native SDK

API Reference

Build iOS applications with Pipecat’s Swift client library

Build Android applications with Pipecat's Kotlin client library

Build native applications with Pipecat’s C++ client library

WebRTC implementation for C++ using Daily

Community

GitHub

Examples

Changelog

Getting Started

Server APIs

Join the Discord

WebRTC transport implementation using Daily for real-time audio/video communication

Daily WebRTC

WebSocket transport implementation for FastAPI web applications

FastAPI WebSocket

WebSocket server transport implementation for real-time audio communication

WebSocket Server

Speech-to-text service implementation using AssemblyAI’s real-time transcription API

AssemblyAI

Speech-to-text service using Azure Cognitive Services Speech SDK

Azure

Speech-to-text service implementation using Deepgram’s real-time transcription API

Deepgram

Speech-to-text service implementation using Gladia’s API

Gladia

Speech-to-text service implementation using Google Cloud’s Speech-to-Text V2 API

Google

Speech-to-text service implementation using Groq’s Whisper API

Groq (Whisper)

Speech-to-text service implementation using OpenAI’s Whisper API

OpenAI (Whisper)

Speech-to-text service implementation using NVIDIA’s Parakeet speech recognition model

NVIDIA Parakeet

Speech-to-text service implementation using a locally-loaded Ultravox multimodal model

Ultravox

Speech-to-text service implementation using locally-downloaded Whisper models

Whisper

Large Language Model service implementation using Anthropic’s Claude API

Anthropic

Large Language Model service implementation using Azure OpenAI API

LLM service implementation using Cerebras’s API with OpenAI-compatible interface

Cerebras

LLM service implementation using DeepSeek’s API with OpenAI-compatible interface

DeepSeek

LLM service implementation using Fireworks AI’s API with OpenAI-compatible interface

Fireworks AI

Large Language Model service implementation using Google’s Gemini API

Google Gemini

LLM service implementation using Grok’s API with OpenAI-compatible interface

Grok

LLM service implementation using Groq’s API with OpenAI-compatible interface

Groq

LLM service implementation using NVIDIA’s NIM (NVIDIA Inference Microservice) API with OpenAI-compatible interface

NVIDIA NIM

LLM service implementation using Ollama with OpenAI-compatible interface

Ollama

Large Language Model services using OpenAI’s chat completion API

OpenAI

LLM service implementation using OpenPipe for LLM request logging and fine-tuning

OpenPipe

LLM service implementation using OpenRouter’s API with OpenAI-compatible interface

OpenRouter

LLM service implementation using Perplexity’s API with OpenAI-compatible interface

Perplexity

LLM service implementation using Together AI’s API with OpenAI-compatible interface

Together AI

Text-to-speech service implementation using Amazon Polly

Amazon Polly

Text-to-speech service using Azure Cognitive Services Speech SDK

Text-to-speech services using Cartesia’s WebSocket and HTTP APIs

Cartesia

Text-to-speech service implementation using Deepgram’s Aura API

Text-to-speech service using ElevenLab’s streaming API with word-level timing

ElevenLabs

Text-to-speech service implementation using NVIDIA’s FastPitch model

NVIDIA FastPitch

Real-time text-to-speech service using Fish Audio's WebSocket API

Fish Audio

Text-to-speech service using Google’s Cloud Text-to-Speech API

Text-to-speech service implementation using LMNT’s streaming API

LMNT

Text-to-speech service using OpenAI’s TTS API

Text-to-speech services using PlayHT’s WebSocket and HTTP APIs

PlayHT

Text-to-speech service implementations using Rime AI

Rime

Text-to-speech service implementation using Coqui’s XTTS streaming server

XTTS

A real-time, multimodal conversational AI service powered by Google’s Gemini

Real-time speech-to-speech service implementation using OpenAI’s Realtime Beta API

OpenAI Realtime Beta

Image generation service implementation using fal’s fast SDXL models

Video service for generating AI avatar responses using Simli's WebRTC API

Simli

Video service implementation for generating AI avatar responses using Tavus

Tavus

Vision service implementation using Moondream for local image analysis and question answering

Moondream

Analytics service for processing and analyzing conversation recordings using Canonical AI’s Voice API

Canonical AI

Performance monitoring integration with Sentry for Pipecat frame processors

Sentry Metrics

Build real-time voice and multimodal applications with Pipecat's RTVI protocol

RTVI (Real-Time Voice Interaction)

Core coordinator for RTVI protocol communication

RTVIProcessor

Specialized processors for handling different aspects of RTVI communication

RTVI Frame Processors

Record and buffer audio from conversations

Audio Recording

Audio noise reduction filter using Koala AI technology from Picovoice

KoalaFilter

Audio noise reduction filter using Krisp AI technology

KrispFilter

Audio noise reduction filter using the noisereduce library

NoisereduceFilter

Voice Activity Detection analyzer using the Silero VAD ONNX model

SileroVADAnalyzer

Audio mixer for combining real-time audio with sound files

SoundfileMixer

Classes and methods for interacting with the Daily API to manage rooms and tokens

Daily REST Helper

Processor for controlling STT muting and interruption handling during bot speech and function calls

Javascript SDK

React SDK

React Native SDK

iOS SDK

Android SDK

C++ SDK

Daily WebRTC Transport

Web

React Native