Overview

SmallWebRTCTransport enables peer-to-peer WebRTC connections between clients and your Pipecat application. It implements bidirectional audio and video streaming using WebRTC for real-time communication.

This transport is intended for lightweight implementations, particularly for local development and testing. It expects your clients to include a corresponding SmallWebRTCTransport implementation. See here for the JavaScript implementation.

SmallWebRTCTransport is best used for testing and development. For production deployments with scale, consider using the DailyTransport, as it has global, low-latency infrastructure.

Installation

To use SmallWebRTCTransport, install the required dependencies:

pip install pipecat-ai[webrtc]

Class Reference

SmallWebRTCConnection

SmallWebRTCConnection manages the WebRTC connection details, peer connection state, and ICE candidates. It handles the signaling process and media tracks.

SmallWebRTCConnection(ice_servers=None)
ice_servers
list[str]

List of STUN/TURN server URLs for ICE connection establishment.

Methods

initialize
async method

Initialize the connection with a client’s SDP offer.

Parameters:

  • sdp: String containing the Session Description Protocol data from client’s offer
  • type: String representing the SDP message type (typically “offer”)
await webrtc_connection.initialize(sdp=client_sdp, type="offer")
connect
async method

Establish the WebRTC peer connection after initialization.

await webrtc_connection.connect()
close
async method

Close the WebRTC peer connection.

await webrtc_connection.close()
renegotiate
async method

Handle connection renegotiation requests.

Parameters:

  • sdp: String containing the Session Description Protocol data for renegotiation
  • type: String representing the SDP message type
  • restart_pc: Boolean indicating whether to completely restart the peer connection (default: False)
await webrtc_connection.renegotiate(sdp=new_sdp, type="offer", restart_pc=False)
get_answer
method

Retrieve the SDP answer to send back to the client.

Returns a dictionary with sdp, type, and pc_id fields.

answer = webrtc_connection.get_answer()
# Returns: {"sdp": "...", "type": "answer", "pc_id": "..."}
send_app_message
method

Send an application message to the client.

Parameters:

  • message: The message to send (will be JSON serialized)
webrtc_connection.send_app_message({"action": "greeting", "text": "Hello!"})
is_connected
method

Check if the connection is active.

if webrtc_connection.is_connected():
    print("Connection is active")
audio_input_track
method

Get the audio input track from the client.

audio_track = webrtc_connection.audio_input_track()
video_input_track
method

Get the video input track from the client.

video_track = webrtc_connection.video_input_track()
ask_to_renegotiate
method

Request the client to initiate connection renegotiation.

webrtc_connection.ask_to_renegotiate()
event_handler
decorator

Register an event handler for connection events.

Events:

  • "app-message": Called when a message is received from the client
  • "connecting": Called when connection is being established
  • "connected": Called when connection is established
  • "disconnected": Called when connection is lost
  • "closed": Called when connection is closed
  • "failed": Called when connection fails
  • "new": Called when a new connection is created
@webrtc_connection.event_handler("connected")
async def on_connected(connection):
    print(f"WebRTC connection established")

SmallWebRTCTransport

SmallWebRTCTransport is the main transport class that manages both input and output transports for WebRTC communication.

SmallWebRTCTransport(
    webrtc_connection: SmallWebRTCConnection,
    params: TransportParams,
    input_name: Optional[str] = None,
    output_name: Optional[str] = None
)
webrtc_connection
SmallWebRTCConnection
required

An instance of SmallWebRTCConnection that manages the WebRTC connection

params
TransportParams
required

Configuration parameters for the transport

input_name
str

Optional name for the input transport

output_name
str

Optional name for the output transport

Methods

input
method

Returns the input transport instance.

input_transport = webrtc_transport.input()
output
method

Returns the output transport instance.

output_transport = webrtc_transport.output()

Event Handlers

on_app_message
async callback

Called when receiving application messages from the client.

Parameters:

  • message: The received message
@webrtc_transport.event_handler("on_app_message")
async def on_app_message(message):
    print(f"Received message: {message}")
on_client_connected
async callback

Called when a client successfully connects.

Parameters:

  • transport: The SmallWebRTCTransport instance
  • webrtc_connection: The connection that was established
@webrtc_transport.event_handler("on_client_connected")
async def on_client_connected(transport, webrtc_connection):
    print(f"Client connected")
on_client_disconnected
async callback

Called when a client disconnects.

Parameters:

  • transport: The SmallWebRTCTransport instance
  • webrtc_connection: The connection that was disconnected
@webrtc_transport.event_handler("on_client_disconnected")
async def on_client_disconnected(transport, webrtc_connection):
    print(f"Client disconnected")
on_client_closed
async callback

Called when a client connection is closed.

Parameters:

  • transport: The SmallWebRTCTransport instance
  • webrtc_connection: The connection that was closed
@webrtc_transport.event_handler("on_client_closed")
async def on_client_closed(transport, webrtc_connection):
    print(f"Client connection closed")

Basic Usage

This basic usage example shows the transport specific parts of a bot.py file required to configure your bot:

from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.pipeline.pipeline import Pipeline
from pipecat.transports.base_transport import TransportParams
from pipecat.transports.network.small_webrtc import SmallWebRTCTransport

async def run_bot(webrtc_connection):
    # Create the WebRTC transport with the provided connection
    transport = SmallWebRTCTransport(
        webrtc_connection=webrtc_connection,
        params=TransportParams(
            audio_in_enabled=True,   # Accept audio from the client
            audio_out_enabled=True,  # Send audio to the client
            vad_enabled=True,        # Enable Voice Activity Detection
            vad_analyzer=SileroVADAnalyzer(),
            vad_audio_passthrough=True,
        ),
    )

    # Set up your services and context

    # Create the pipeline
    pipeline = Pipeline([
        transport.input(),              # Receive audio from client
        stt,                            # Convert speech to text
        context_aggregator.user(),      # Add user messages to context
        llm,                            # Process text with LLM
        tts,                            # Convert text to speech
        transport.output(),             # Send audio responses to client
        context_aggregator.assistant(), # Add assistant responses to context
    ])

    # Register event handlers
    @transport.event_handler("on_client_connected")
    async def on_client_connected(transport, client):
        logger.info("Client connected")
        # Start the conversation when client connects
        await task.queue_frames([context_aggregator.user().get_context_frame()])

    @transport.event_handler("on_client_disconnected")
    async def on_client_disconnected(transport, client):
        logger.info("Client disconnected")

    @transport.event_handler("on_client_closed")
    async def on_client_closed(transport, client):
        logger.info("Client closed")
        await task.cancel()

How to connect with SmallWebRTCTransport

For a client/server connection, you have two options for how to connect the client to the server:

  1. Use a Pipecat client SDK with the SmallWebRTCTransport. See the Client SDK docs to get started.
  2. Using the WebRTC API directly. This is only recommended for advanced use cases where the Pipecat client SDKs don’t have an available transport.

Examples

To see a complete implementation, check out the following examples:

Media Handling

Audio

Audio is processed in 20ms chunks by default. The transport handles audio format conversion and resampling as needed:

  • Input audio is processed at 16kHz (mono) to be compatible with speech recognition services
  • Output audio can be configured to match your application’s requirements, but it must be mono, 16-bit PCM audio

Video

Video is streamed using RGB format by default. The transport provides:

  • Frame conversion between different color formats (RGB, YUV, etc.)
  • Configurable resolution and framerate

Advanced Configuration

ICE Servers

For better connectivity, especially when testing across different networks, you can provide STUN servers:

webrtc_connection = SmallWebRTCConnection(
    ice_servers=["stun:stun.l.google.com:19302", "stun:stun1.l.google.com:19302"]
)

Troubleshooting

If clients have trouble connecting or streaming:

  1. Check browser console for WebRTC errors
  2. Ensure you’re using HTTPS in production (required for WebRTC)
  3. For testing across networks, consider using Daily which provides TURN servers
  4. Verify browser permissions for camera and microphone