Pipecat provides client SDKs for multiple platforms, all implementing the RTVI (Real-Time Voice and Video Inference) standard. These SDKs make it easy to build real-time AI applications that can handle voice, video, and text interactions.
Core Functionality All Pipecat client SDKs provide:
Media Management Handle device inputs and media streams for audio and video
Bot Integration Configure and communicate with your Pipecat bot
Action Handling Send actions and process bot responses
Session Management Manage connection state and error handling
Interaction Modes Real-time Streaming Single-turn Actions Establish ongoing connections via WebSocket or WebRTC for:
Live voice conversations
Real-time video processing
Continuous interactions
// Example: Establishing a real-time connection import { RTVIEvent , RTVIMessage , RTVIClient } from "realtime-ai" ; import { DailyTransport } from "@pipecat-ai/daily-transport" ; const rtviClient = new RTVIClient ({ transport: new DailyTransport (), params: { baseUrl: "https://your-connect-end-point-here" , endpoint: { connect: "/connect" , } } enableMic : true , enableCam: false , enableScreenShare: false , timeout: 15 * 1000 , callbacks: { onBotConnected : () => { console . log ( "[CALLBACK] Bot connected" ); }, onBotDisconnected : () => { console . log ( "[CALLBACK] Bot disconnected" ); }, onBotReady : () => { console . log ( "[CALLBACK] Bot ready to chat!" ); }, }, }); try { await rtviClient . connect (); } catch ( e ) { console . error ( e . message ); } // Events rtviClient . on ( RTVIEvent . Connected , () => { console . log ( "[EVENT] User connected" ); }); rtviClient . on ( RTVIEvent . Disconnected , () => { console . log ( "[EVENT] User disconnected" ); }); Establish ongoing connections via WebSocket or WebRTC for:
Live voice conversations
Real-time video processing
Continuous interactions
// Example: Establishing a real-time connection import { RTVIEvent , RTVIMessage , RTVIClient } from "realtime-ai" ; import { DailyTransport } from "@pipecat-ai/daily-transport" ; const rtviClient = new RTVIClient ({ transport: new DailyTransport (), params: { baseUrl: "https://your-connect-end-point-here" , endpoint: { connect: "/connect" , } } enableMic : true , enableCam: false , enableScreenShare: false , timeout: 15 * 1000 , callbacks: { onBotConnected : () => { console . log ( "[CALLBACK] Bot connected" ); }, onBotDisconnected : () => { console . log ( "[CALLBACK] Bot disconnected" ); }, onBotReady : () => { console . log ( "[CALLBACK] Bot ready to chat!" ); }, }, }); try { await rtviClient . connect (); } catch ( e ) { console . error ( e . message ); } // Events rtviClient . on ( RTVIEvent . Connected , () => { console . log ( "[EVENT] User connected" ); }); rtviClient . on ( RTVIEvent . Disconnected , () => { console . log ( "[EVENT] User disconnected" ); }); Send individual actions for:
Text-to-text chat
One-off image processing
Batch operations
// Example: Sending a single action import { RTVIClient } from "realtime-ai" ; const rtviClient = new RTVIClient ({ params: { baseUrl: "https://your-connect-end-point-here" , endpoint: { connect: "/connect" , action: "/action" , }, }, callbacks: { onBotText : ( text : BotLLMTextData ) => { console . log ( "Text response:" , text ); }, }, }); try { await rtviClient . action ({ service: "llm" , action: "append_to_messages" , arguments: [ { name: "messages" , value: [ { role: "user" , content: "tell me a joke" , }, ], }, ], }); } catch ( e ) { console . error ( e . message ); } About RTVI Pipecat’s client SDKs implement the RTVI (Real-Time Voice and Video Inference) standard, an open specification for real-time AI inference. This means:
Your code can work with any RTVI-compatible inference service
You get battle-tested tooling for real-time multimedia handling
You can easily set up development and testing environments
For production deployments, we recommend setting up your own authentication
endpoints rather than exposing service credentials directly in client
applications.
Next Steps Get started by trying out examples:
