SmallWebRTCTransport
A lightweight WebRTC transport for peer-to-peer audio and video communication in Pipecat
Overview
SmallWebRTCTransport
enables peer-to-peer WebRTC connections between clients and your Pipecat application. It implements bidirectional audio and video streaming using WebRTC for real-time communication.
This transport is intended for lightweight implementations, particularly for local development and testing. It expects your clients to include a corresponding SmallWebRTCTransport
implementation. See here for the JavaScript implementation.
SmallWebRTCTransport
is best used for testing and development. For
production deployments with scale, consider using the
DailyTransport, as it has global,
low-latency infrastructure.
Installation
To use SmallWebRTCTransport
, install the required dependencies:
Class Reference
SmallWebRTCConnection
SmallWebRTCConnection
manages the WebRTC connection details, peer connection state, and ICE candidates. It handles the signaling process and media tracks.
List of STUN/TURN server URLs for ICE connection establishment.
Methods
Initialize the connection with a client’s SDP offer.
Parameters:
sdp
: String containing the Session Description Protocol data from client’s offertype
: String representing the SDP message type (typically “offer”)
Establish the WebRTC peer connection after initialization.
Close the WebRTC peer connection.
Handle connection renegotiation requests.
Parameters:
sdp
: String containing the Session Description Protocol data for renegotiationtype
: String representing the SDP message typerestart_pc
: Boolean indicating whether to completely restart the peer connection (default: False)
Retrieve the SDP answer to send back to the client.
Returns a dictionary with sdp
, type
, and pc_id
fields.
Send an application message to the client.
Parameters:
message
: The message to send (will be JSON serialized)
Check if the connection is active.
Get the audio input track from the client.
Get the video input track from the client.
Request the client to initiate connection renegotiation.
Register an event handler for connection events.
Events:
"app-message"
: Called when a message is received from the client"connecting"
: Called when connection is being established"connected"
: Called when connection is established"disconnected"
: Called when connection is lost"closed"
: Called when connection is closed"failed"
: Called when connection fails"new"
: Called when a new connection is created
SmallWebRTCTransport
SmallWebRTCTransport
is the main transport class that manages both input and output transports for WebRTC communication.
An instance of SmallWebRTCConnection
that manages the WebRTC connection
Configuration parameters for the transport
Optional name for the input transport
Optional name for the output transport
Methods
Returns the input transport instance.
Returns the output transport instance.
Event Handlers
Called when receiving application messages from the client.
Parameters:
message
: The received message
Called when a client successfully connects.
Parameters:
transport
: The SmallWebRTCTransport instancewebrtc_connection
: The connection that was established
Called when a client disconnects.
Parameters:
transport
: The SmallWebRTCTransport instancewebrtc_connection
: The connection that was disconnected
Called when a client connection is closed.
Parameters:
transport
: The SmallWebRTCTransport instancewebrtc_connection
: The connection that was closed
Basic Usage
This basic usage example shows the transport specific parts of a bot.py file required to configure your bot:
How to connect with SmallWebRTCTransport
For a client/server connection, you have two options for how to connect the client to the server:
- Use a Pipecat client SDK with the
SmallWebRTCTransport
. See the Client SDK docs to get started. - Using the WebRTC API directly. This is only recommended for advanced use cases where the Pipecat client SDKs don’t have an available transport.
Examples
To see a complete implementation, check out the following examples:
Video Transform
Demonstrates real-time video processing using WebRTC transport
Voice Agent
Implements a voice assistant using WebRTC for audio communication
Media Handling
Audio
Audio is processed in 20ms chunks by default. The transport handles audio format conversion and resampling as needed:
- Input audio is processed at 16kHz (mono) to be compatible with speech recognition services
- Output audio can be configured to match your application’s requirements, but it must be mono, 16-bit PCM audio
Video
Video is streamed using RGB format by default. The transport provides:
- Frame conversion between different color formats (RGB, YUV, etc.)
- Configurable resolution and framerate
Advanced Configuration
ICE Servers
For better connectivity, especially when testing across different networks, you can provide STUN servers:
Troubleshooting
If clients have trouble connecting or streaming:
- Check browser console for WebRTC errors
- Ensure you’re using HTTPS in production (required for WebRTC)
- For testing across networks, consider using Daily which provides TURN servers
- Verify browser permissions for camera and microphone