Overview

Transports

More Information

Community

GitHub

Examples

Changelog

Pipecat

Getting Started

Guides

Server APIs

Client SDKs

Join the Discord

RealTimeWebSocketTransport

Pipecat is a framework for building voice-enabled, real-time, multimodal AI applications

Get Pipecat and its required services installed on your machine

Installation & Setup

Build and run your first Pipecat application

Quickstart

Understanding how Pipecat works: frames, processors, and pipelines.

Core Concepts

Common applications you can build with Pipecat

Use Cases

Learn how to deploy, scale, and extend your Pipecat applications

Enable LLMs to interact with external services and APIs

Function Calling

Create a real-time AI chatbot using Gemini Multimodal Live and Pipecat

Gemini Multimodal Live

Building with Gemini Multimodal Live

Learn how to monitor performance and LLM/TTS usage with Pipecat.

Metrics

Learn how to integrate Krisp noise cancellation into your Pipecat application

Noise cancellation with Krisp

Learn how to create structured conversations using Pipecat’s flow system

Pipecat Flows

How to call your bots (or have them call you)

Dial-in: WebRTC (Daily)

Call your Pipecat bot using Daily WebRTC and a Twilio phone number

Dial-in: WebRTC (Twilio + Daily)

Call your Pipecat bot over websockets using Twilio

Dial-in: WebSockets (Twilio)

Dialout: WebRTC (Daily)

Basic deployment pattern for Pipecat bots

Deployment pattern

Example: Fly.io

Example: Cerebrium

Complete reference for Pipecat’s Python server APIs and services

Server API Reference

Reference docs

AI services integrated with Pipecat and their setup requirements

Supported Services

WebRTC transport implementation using Daily for real-time audio/video communication

Daily WebRTC

Speech-to-text service implementation using AssemblyAI’s real-time transcription API

AssemblyAI

Speech-to-text service using Azure Cognitive Services Speech SDK

Azure

Speech-to-text service implementation using Deepgram’s real-time transcription API

Deepgram

Speech-to-text service implementation using Gladia’s API

Gladia

Speech-to-text service implementation using locally-downloaded Whisper models

Whisper

Large Language Model service implementation using Anthropic’s Claude API

Anthropic

Large Language Model service implementation using Azure OpenAI API

LLM service implementation using Fireworks AI’s API with OpenAI-compatible interface

Fireworks AI

Large Language Model service implementation using Google’s Gemini API

Google Gemini

LLM service implementation using Grok’s API with OpenAI-compatible interface

Grok

LLM service implementation using Groq’s API with OpenAI-compatible interface

Groq

LLM service implementation using NVIDIA’s NIM (NVIDIA Inference Microservice) API with OpenAI-compatible interface

NVIDIA NIM

LLM service implementation using Ollama with OpenAI-compatible interface

Ollama

Large Language Model services using OpenAI’s chat completion API

OpenAI

LLM service implementation using OpenPipe for LLM request logging and fine-tuning

OpenPipe

LLM service implementation using Together AI’s API with OpenAI-compatible interface

Together AI

Text-to-speech service implementation using Amazon AWS Polly

AWS Polly

Text-to-speech service using Azure Cognitive Services Speech SDK

Text-to-speech services using Cartesia’s WebSocket and HTTP APIs

Cartesia

Text-to-speech service implementation using Deepgram’s Aura API

Text-to-speech service using ElevenLab’s streaming API with word-level timing

ElevenLabs

Text-to-speech service using Google’s Cloud Text-to-Speech API

Google

Text-to-speech service implementation using LMNT’s streaming API

LMNT

Text-to-speech service using OpenAI’s TTS API

Text-to-speech services using PlayHT’s WebSocket and HTTP APIs

PlayHT

Text-to-speech service implementation using Rime AI

Rime

Text-to-speech service implementation using Coqui’s XTTS streaming server

XTTS

A real-time, multimodal conversational AI service powered by Google’s Gemini

Real-time speech-to-speech service implementation using OpenAI’s Realtime Beta API

OpenAI Realtime Beta

Image generation service implementation using fal’s fast SDXL models

Video service for generating AI avatar responses using Simli's WebRTC API

Simli

Video service implementation for generating AI avatar responses using Tavus

Tavus

Vision service implementation using Moondream for local image analysis and question answering

Moondream

Analytics service for processing and analyzing conversation recordings using Canonical AI’s Voice API

Canonical AI

Performance monitoring integration with Sentry for Pipecat frame processors

Sentry Metrics

Overview of Pipecat’s core service implementations

Base implementations for handling audio/video I/O streams

Transport

Base implementations for speech-to-text and text-to-speech services

Speech Service

Base implementation for Large Language Model services

LLM Service

Base implementations for text filtering and preprocessing

Text Processing

Base implementations for image and vision services

Media Service

Build real-time voice and multimodal applications with Pipecat's RTVI protocol

RTVI (Real-Time Voice Interaction)

Core coordinator for RTVI protocol communication

RTVIProcessor

Specialized processors for handling different aspects of RTVI communication

RTVI Frame Processors

Technical reference for Pipecat’s conversation flow system

Audio noise reduction filter using Krisp AI technology

KrispFilter

Audio noise reduction filter using the noisereduce library

NoisereduceFilter

Voice Activity Detection analyzer using the Silero VAD ONNX model

SileroVADAnalyzer

Audio mixer for combining real-time audio with sound files

SoundfileMixer

Classes and methods for interacting with the Daily API to manage rooms and tokens

Daily REST Helper

Processor for controlling STT muting and interruption handling during bot speech and function calls

STTMuteFilter

Client libraries for building real-time AI applications with Pipecat

Build web applications with Pipecat’s JavaScript client library

SDK Introduction

Client Constructor

Client Methods

Selecting and authenticating bot services

Services

Passing service configuration values to a bot.

Configuration

Callbacks and events

Actions

Messages

Errors

Introduction

LLM Helper

Transport Overview

Daily WebRTC Transport

GeminiLiveWebSocketTransport

Build React applications with Pipecat's React client library

Ready-to-use React components for Pipecat applications

Components

React hooks for accessing Pipecat client functionality

Hooks

Build iOS applications with Pipecat’s Swift client library

API Reference

WebRTC implementation for iOS using Daily

Build Android applications with Pipecat's Kotlin client library

WebRTC implementation for Android using Daily

Build native applications with Pipecat’s C++ client library

WebRTC implementation for C++ using Daily

Javascript SDK

React SDK

iOS SDK

Android SDK

C++ SDK

RealTimeWebSocketTransport

Overview

Transports

More Information

Source

Package

Javascript SDK

React SDK

iOS SDK

Android SDK

C++ SDK

​Overview

​Transports

​More Information

Source

Package

Overview

Transports

More Information