SmallWebRTCTransport

​

Overview

SmallWebRTCTransport enables peer-to-peer WebRTC connections between clients and your Pipecat application. It implements bidirectional audio and video streaming using WebRTC for real-time communication.

This transport is intended for lightweight implementations, particularly for local development and testing. It expects your clients to include a corresponding SmallWebRTCTransport implementation. See here for the JavaScript implementation.

​

Installation

To use SmallWebRTCTransport, install the required dependencies:

pip install pipecat-ai[webrtc]

​

Class Reference

​

SmallWebRTCConnection

SmallWebRTCConnection manages the WebRTC connection details, peer connection state, and ICE candidates. It handles the signaling process and media tracks.

SmallWebRTCConnection(ice_servers=None)

​

Methods

await webrtc_connection.initialize(sdp=client_sdp, type="offer")

await webrtc_connection.connect()

await webrtc_connection.close()

await webrtc_connection.disconnect()

await webrtc_connection.renegotiate(sdp=new_sdp, type="offer", restart_pc=False)

answer = webrtc_connection.get_answer()
# Returns: {"sdp": "...", "type": "answer", "pc_id": "..."}

webrtc_connection.send_app_message({"action": "greeting", "text": "Hello!"})

if webrtc_connection.is_connected():
    print("Connection is active")

audio_track = webrtc_connection.audio_input_track()

video_track = webrtc_connection.video_input_track()

webrtc_connection.replace_audio_track(new_audio_track)

webrtc_connection.replace_video_track(new_video_track)

webrtc_connection.ask_to_renegotiate()

@webrtc_connection.event_handler("connected")
async def on_connected(connection):
    print(f"WebRTC connection established")

​

SmallWebRTCTransport

SmallWebRTCTransport is the main transport class that manages both input and output transports for WebRTC communication.

SmallWebRTCTransport(
    webrtc_connection: SmallWebRTCConnection,
    params: TransportParams,
    input_name: Optional[str] = None,
    output_name: Optional[str] = None
)

​

Methods

input_transport = webrtc_transport.input()

output_transport = webrtc_transport.output()

await webrtc_transport.send_image(image_frame)

await webrtc_transport.send_audio(audio_frame)

​

Event Handlers

@webrtc_transport.event_handler("on_app_message")
async def on_app_message(message):
    print(f"Received message: {message}")

@webrtc_transport.event_handler("on_client_connected")
async def on_client_connected(transport, webrtc_connection):
    print(f"Client connected")

@webrtc_transport.event_handler("on_client_disconnected")
async def on_client_disconnected(transport, webrtc_connection):
    print(f"Client disconnected")

@webrtc_transport.event_handler("on_client_closed")
async def on_client_closed(transport, webrtc_connection):
    print(f"Client connection closed")

​

Basic Usage

This basic usage example shows the transport specific parts of a bot.py file required to configure your bot:

from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.pipeline.pipeline import Pipeline
from pipecat.transports.base_transport import TransportParams
from pipecat.transports.network.small_webrtc import SmallWebRTCTransport
from pipecat.transports.network.webrtc_connection import SmallWebRTCConnection

async def run_bot(webrtc_connection):
    # Create the WebRTC transport with the provided connection
    transport = SmallWebRTCTransport(
        webrtc_connection=webrtc_connection,
        params=TransportParams(
            audio_in_enabled=True,   # Accept audio from the client
            audio_out_enabled=True,  # Send audio to the client
            vad_analyzer=SileroVADAnalyzer(),
        ),
    )

    # Set up your services and context

    # Create the pipeline
    pipeline = Pipeline([
        transport.input(),              # Receive audio from client
        stt,                            # Convert speech to text
        context_aggregator.user(),      # Add user messages to context
        llm,                            # Process text with LLM
        tts,                            # Convert text to speech
        transport.output(),             # Send audio responses to client
        context_aggregator.assistant(), # Add assistant responses to context
    ])

    # Register event handlers
    @transport.event_handler("on_client_connected")
    async def on_client_connected(transport, client):
        logger.info("Client connected")
        # Start the conversation when client connects
        await task.queue_frames([context_aggregator.user().get_context_frame()])

    @transport.event_handler("on_client_disconnected")
    async def on_client_disconnected(transport, client):
        logger.info("Client disconnected")

    @transport.event_handler("on_client_closed")
    async def on_client_closed(transport, client):
        logger.info("Client closed")
        await task.cancel()

​

How to connect with SmallWebRTCTransport

For a client/server connection, you have two options for how to connect the client to the server:

1. Use a Pipecat client SDK with the SmallWebRTCTransport. See the Client SDK docs to get started.
2. Using the WebRTC API directly. This is only recommended for advanced use cases where the Pipecat client SDKs don’t have an available transport.

​

Examples

To see a complete implementation, check out the following examples:

​

Media Handling

​

Audio

Audio is processed in 20ms chunks by default. The transport handles audio format conversion and resampling as needed:

• Input audio is processed at 16kHz (mono) to be compatible with speech recognition services
• Output audio can be configured to match your application’s requirements, but it must be mono, 16-bit PCM audio

​

Video

Video is streamed using RGB format by default. The transport provides:

• Frame conversion between different color formats (RGB, YUV, etc.)
• Configurable resolution and framerate

​

WebRTC ICE Servers Configuration

When implementing WebRTC in your project, STUN (Session Traversal Utilities for NAT) and TURN (Traversal Using Relays around NAT) servers are usually needed in cases where users are behind routers or firewalls.

In local networks (e.g., testing within the same home or office network), you usually don’t need to configure STUN or TURN servers. In such cases, WebRTC can often directly establish peer-to-peer connections without needing to traverse NAT or firewalls.

​

What are STUN and TURN Servers?

• STUN Server: Helps clients discover their public IP address and port when they’re behind a NAT (Network Address Translation) device (like a router). This allows WebRTC to attempt direct peer-to-peer communication by providing the public-facing IP and port.

• TURN Server: Used as a fallback when direct peer-to-peer communication isn’t possible due to strict NATs or firewalls blocking connections. The TURN server relays media traffic between peers.

​

Why are ICE Servers Important?

ICE (Interactive Connectivity Establishment) is a framework used by WebRTC to handle network traversal and NAT issues. The iceServers configuration provides a list of STUN and TURN servers that WebRTC uses to find the best way to connect two peers.

​

Advanced Configuration

​

ICE Servers

For better connectivity, especially when testing across different networks, you can provide STUN servers:

webrtc_connection = SmallWebRTCConnection(
    ice_servers=["stun:stun.l.google.com:19302", "stun:stun1.l.google.com:19302"]
)

You can also use IceServer objects for more advanced configuration:

from pipecat.transports.network.webrtc_connection import IceServer

webrtc_connection = SmallWebRTCConnection(
    ice_servers=[
        IceServer(urls="stun:stun.l.google.com:19302"),
        IceServer(
            urls="turn:turn.example.com:3478",
            username="username",
            credential="password"
        )
    ]
)

​

Troubleshooting

If clients have trouble connecting or streaming:

1. Check browser console for WebRTC errors
2. Ensure you’re using HTTPS in production (required for WebRTC)
3. For testing across networks, consider using Daily which provides TURN servers
4. Verify browser permissions for camera and microphone