Skip to main content

​
Overview

Learn how to build web and mobile applications using Pipecat’s client/server architecture. This approach separates your bot logic from your user interface, giving you full control over the client experience while maintaining real-time voice communication. This guide introduces two key technologies that power client/server applications:

​
Pipecat Client SDKs

Pipecat provides client SDKs for multiple platforms, all implementing the RTVI (Real-Time Voice and Video Inference) standard. These SDKs handle the complex real-time communication between your custom interface and your Pipecat bot.

JavaScript

Build web applications with vanilla JavaScript

React

Create React applications with hooks and components

React Native

Build mobile apps with React Native

Swift

Native iOS applications

Kotlin

Native Android applications

C++

High-performance native applications
Key Benefits:
  • Real-time communication: Handle voice, video, and messaging in real-time
  • RTVI standard: Open specification ensuring compatibility and consistency
  • Cross-platform: Consistent API across all supported platforms
  • Production-ready: Battle-tested for multimedia handling

​
Voice UI Kit

The Voice UI Kit provides React components and templates for quickly building voice AI interfaces. It’s designed specifically to accelerate Pipecat application development. Key Features:
  • Debug console: Flexible UI for testing and benchmarking your bots
  • Headless components: Building blocks for custom voice interfaces
  • Drop-in templates: Fully-featured UIs ready for development and demos
  • Fully customizable: Built on Tailwind with CSS variables for easy theming
  • Responsive design: Optimized for desktop, tablet, and mobile

​
React + SmallWebRTC Example

This example demonstrates a React client using the Voice UI Kit’s console template, connected to a Pipecat bot server via SmallWebRTC transport.

​
Prerequisites

Environment
  • Python 3.10 or later
  • uv package manager installed
  • Node.js 16 or later
AI Service API Keys This quickstart uses three AI services working together in a pipeline. You’ll need API keys from each service:

Deepgram (STT)

Create an account and generate your API key for real-time speech recognition.

OpenAI (LLM)

Create an account and generate an API key for intelligent conversation responses.

Cartesia (TTS)

Sign up and generate your API key for natural voice synthesis.
Have these API keys ready. You’ll add them to your environment file in the next section.

​
Setup

This example requires running both a server and client in two separate terminal windows.

​
1. Clone the quickstart repository

git clone https://github.com/pipecat-ai/pipecat-quickstart-client-server.git
cd pipecat-quickstart-client-server

​
2. Set up the server (Terminal 1)

Navigate to the server directory and set up your environment: 
cd server
Create your environment file: 
cp env.example .env
Open the .env file and add your API keys: 
DEEPGRAM_API_KEY=your_deepgram_api_key
OPENAI_API_KEY=your_openai_api_key
CARTESIA_API_KEY=your_cartesia_api_key
Set up your virtual environment and install dependencies: 
uv sync
Run your bot server: 
uv run bot.py
You should see output confirming your server is running and ready to receive connections.

​
3. Set up the client (Terminal 2)

In a new terminal window, navigate to the client directory: 
cd client
Install client dependencies: 
npm install
Start the development server: 
npm run dev
You should see output like: 

  VITE v6.2.5  ready in 5957 ms

  ➜  Local:   http://localhost:5173/
  ➜  Network: use --host to expose
  ➜  press h + enter to show help

​
4. Connect and test

Open http://localhost:5173 in your browser. You’ll see the Voice UI Kit console interface with a connect button. Click Connect and allow microphone access when prompted. The console will establish a connection to your bot server and you can start having a voice conversation!

​
Understanding the Architecture

​
Transport Layer

Both the client and server use the SmallWebRTC transport, which enables:
  • Bidirectional audio communication: Real-time voice data flows between client and server
  • Bidirectional events and messaging: RTVI protocol messages for coordination and control
  • Synchronized state: Client and server stay in sync throughout the conversation

​
Client Implementation

The React client uses the Voice UI Kit’s ConsoleTemplate component: 
import {
  ConsoleTemplate,
  FullScreenContainer,
  ThemeProvider,
} from "@pipecat-ai/voice-ui-kit";

createRoot(document.getElementById("root")!).render(
  <StrictMode>
    <ThemeProvider>
      <FullScreenContainer>
        <ConsoleTemplate
          transportType="smallwebrtc"
          connectParams={{
            connectionUrl: "/api/offer",
          }}
        />
      </FullScreenContainer>
    </ThemeProvider>
  </StrictMode>
);
Key Points:
  • Full-page experience: The console template provides a complete interface for voice interaction
  • Custom UI building: The Voice UI Kit also offers individual components for building custom interfaces
  • Transport configuration: The client connects to the /api/offer endpoint provided by the server

​
Server Implementation

The server uses Pipecat’s development runner, which automatically provides:
  • FastAPI server: Handles HTTP requests and WebRTC connections
  • WebRTC endpoint: The /api/offer endpoint that clients connect to
  • Connection management: Automatic setup and teardown of client connections
This is the same server-side Pipecat bot from the Quickstart example.

​
Building for Other Platforms

This example uses React and SmallWebRTC, but Pipecat’s client SDKs work across platforms with the same server-side bot code. The RTVI standard ensures consistent communication regardless of which client SDK you use.

Multi-Platform Example

See the same bot working with 6 different client implementations: JavaScript, React, iOS (Swift), Android (Kotlin), React Native, and Daily Prebuilt
 Key advantages:
  • Write once, deploy everywhere: Your Pipecat server works with any client platform
  • Consistent experience: RTVI protocol ensures uniform behavior across platforms
  • Easy migration: Switch between client platforms without changing server code

​
Deployment Considerations

​
Local Development

  • Client connects to local server: The development runner provides all necessary endpoints
  • Automatic setup: No additional configuration needed beyond API keys

​
Pipecat Cloud Deployment

  • Server: Deploy your bot code as-is to Pipecat Cloud
  • Client: Update connection endpoint to your web server, which then connects to Pipecat Cloud’s /{agent}/start endpoint
  • API key security: Your web server keeps API keys secret from the client

​
Self-Hosting

  • Custom server setup: Implement your own FastAPI or similar server
  • Connection handling: Handle bot startup and WebRTC connection management
  • Similar pattern: Web server receives client connections, then starts bot instances
A web server layer is required in production to keep your AI service API keys secure and never expose them to client-side code.

​
Next Steps

Now that you have a working client/server application:
  • Customize the interface: Explore Voice UI Kit components to build custom interfaces beyond the console template
  • Try other platforms: Check out the iOS, Android, and React Native client SDKs
  • Join the community: Connect with other developers on Discord to share your projects and get help