WebSocket Transport

WebSocketTransport enables a purely WebSocket based connection between clients and your Pipecat application. It implements bidirectional audio streaming using a WebSocket for real-time communication. This transport is intended for lightweight implementations, particularly for local development and testing. It expects your Pipecat server to include the corresponding WebSocketTransport server-side implementation.

The WebSocketTransport is best suited for server-server applications and prototyping client/server apps.For client/server production applications, we strongly recommend using a WebRTC-based transport for robust network and media handling. For more on WebRTC vs. Websocket communication, check out this article.

Usage

Basic Setup

import { PipecatClient } from "@pipecat-ai/client-js";
import {
  WebSocketTransport,
  ProtobufFrameSerializer,
} from "@pipecat-ai/websocket-transport";

const pcClient = new PipecatClient({
  transport: new WebSocketTransport({
    serializer: new ProtobufFrameSerializer(),
    recorderSampleRate: 8000,
    playerSampleRate: 8000,
  }),
  enableCam: false, // Default camera off
  enableMic: true, // Default microphone on
  callbacks: {
    // Event handlers
  },
});

await pcClient.connect({
  wsUrl: "ws://localhost:7860/ws", // Your WebSocket server URL
});

API Reference

Constructor Options

type WebSocketTransportOptions = {
  /** @deprecated Use wsUrl instead */
  ws_url?: string;
  wsUrl?: string;
  token?: string;
};

export interface WebSocketTransportConstructorOptions extends WebSocketTransportOptions {
  mediaManager?: MediaManager;
  serializer?: WebSocketSerializer;
  recorderSampleRate?: number;
  playerSampleRate?: number;
}

Properties

TransportConnectionParams

connect() accepts WebSocketTransportOptions: wsUrl (or the deprecated ws_url) and token. wsUrl overrides the value set in the constructor. token is appended to the URL as a URL-encoded token query parameter; it is read only from connect(), so a token passed to the constructor is ignored. Any other key raises an RTVIError, so the serializer, media manager and sample rates are constructor-only.

pcClient.connect({
  wsUrl: 'ws://localhost:7860/ws'
});
// OR...
pcClient.startBotAndConnect({
  endpoint: '/api/start', // returns { wsUrl }
});

# See
# https://github.com/pipecat-ai/pipecat-examples/tree/main/websocket
# for a complete WebSocket client and server example.
@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
    await websocket.accept()
    print("WebSocket connection accepted")
    try:
        await run_bot(websocket)
    except Exception as e:
        print(f"Exception in run_bot: {e}")


@app.post("/api/start")
async def start(request: Request) -> Dict[Any, Any]:
    ws_url = "ws://localhost:7860/ws"
    return {"wsUrl": ws_url}

# See
# https://github.com/pipecat-ai/pipecat-examples/blob/main/websocket/bot.py
# for a complete example of a bot script using the WebSocketTransport.
from pipecat.serializers.protobuf import ProtobufFrameSerializer
from pipecat.transports.websocket.fastapi import (
    FastAPIWebsocketParams,
    FastAPIWebsocketTransport,
)

async def run_bot(websocket_client):
    ws_transport = FastAPIWebsocketTransport(
        websocket=websocket_client,
        params=FastAPIWebsocketParams(
            audio_in_enabled=True,
            audio_out_enabled=True,
            add_wav_header=False,
            serializer=ProtobufFrameSerializer(),
        ),
    )

    llm = ... # Initialize your LLM here, e.g., OpenAI, HuggingFace, etc.

    messages = [{"role": "system", "content": "You are a helpful assistant."}]
    context = LLMContext(messages)
    user_aggregator, assistant_aggregator = LLMContextAggregatorPair(
        context,
        user_params=LLMUserAggregatorParams(
            vad_analyzer=SileroVADAnalyzer(),
        ),
    )

    pipeline = Pipeline(
        [
            ws_transport.input(),
            user_aggregator,
            llm,  # LLM
            ws_transport.output(),
            assistant_aggregator,
        ]
    )

    worker = PipelineWorker(
        pipeline,
        params,
    )
    ...

Methods

For most operations, you will not interact with the transport directly. Most methods have an equivalent in the PipecatClient and should be called from the PipecatClient. However, there are a few transport-specific methods that you may need to call directly. When doing so, be sure to access your transport via the transport property of the PipecatClient instance.

method

Sends an ArrayBuffer of user audio to the bot through the serializer, queuing it until the transport is ready. The transport registers this method as the media manager’s audio callback, so microphone audio is sent without calling it; call it only for audio from another source.

transport.handleUserAudioStream(chunk.data);

method

Sends a message that is not an RTVI message, encoded with the serializer’s serialize(). Use it to send Twilio media stream events, such as connected and start, with TwilioSerializer.

transport.sendRawMessage({
  event: "connected",
  protocol: "Call",
  version: "1.0.0",
});

Media Support

WebSocketTransport carries audio only. cam, screenShare, and botVideo are false, and mic follows the configured media manager. See mediaSupport.

Events

The transport implements the various PipecatClient event handlers.

Reconnection Handling

WebSocketTransport does not reconnect. If the WebSocket connection closes unexpectedly, the transport reports a fatal error through onError and the PipecatClient disconnects.

More Information

WebSocket Demo

Basic Agent example using a WebSocket transport

Twilio Demo

Example using a WebSocket transport to simulate a Twilio connection to a bot

Source

WebSocketTransport

Package

@pipecat-ai/websocket-transport

Get Started

Standards

Pipecat Server

Client SDKs

Pipecat Flows

Pipecat Cloud

CLI

Pipecat Context Hub

Usage

Basic Setup

API Reference

Constructor Options

Properties

TransportConnectionParams

Methods

Media Support

Events

Reconnection Handling

More Information

WebSocket Demo

Twilio Demo

Source

Package

​Usage

​Basic Setup

​API Reference

​Constructor Options

​Properties

​TransportConnectionParams

​Methods

​Media Support

​Events

​Reconnection Handling

​More Information

WebSocket Demo

Twilio Demo

Source

Package

Usage

Basic Setup

API Reference

Constructor Options

Properties

TransportConnectionParams

Methods

Media Support

Events

Reconnection Handling

More Information