​
Overview

The OpenAIRealTimeWebRTCTransport is a fully functional RTVI Transport. It provides a framework for implementing real-time communication directly with the OpenAI Realtime API using WebRTC voice-to-voice service. It handles media device management, audio/video streams, and state management for the connection.

Transports of this type are design primarily for development and testing purposes. For production applications, you will need to build a server component with a server-friendly transport, like the DailyTransport, to securely handle API keys.

​
Usage

​
Basic Setup

import { OpenAIRealTimeWebRTCTransport, OpenAIServiceOptions } from '@pipecat-ai/openai-realtime-webrtc-transport';

const options: OpenAIServiceOptions = {
  api_key: 'YOUR_API_KEY',
  session_config: {
    instructions: 'you are a confused jellyfish',
  }
};

const transport = new OpenAIRealTimeWebRTCTransport(options);
let RTVIConfig: RTVIClientOptions = {
  transport,
  ...
};

​
Configuration Options

interface OpenAIServiceOptions {
  api_key: string;                    // Required: Your OpenAI API key
  initial_messages?: Array<{          // Optional: Initial conversation context
    content: string;
    role: string;
  }>;
  session_config?: {
    modailities?: string;
    instructions?: string;
    voice?:
      | "alloy"
      | "ash"
      | "ballad"
      | "coral"
      | "echo"
      | "sage"
      | "shimmer"
      | "verse";
    input_audio_transcription?: {
      model: "whisper-1";
    };
    temperature?: number;
    max_tokens?: number | "inf";
  };
}

​
Sending Messages

// at setup time...
llmHelper = new LLMHelper({});
rtviClient.registerHelper("llm", llmHelper);
// the 'llm' name in this call above isn't used.
//that value is specific to working with a pipecat pipeline

// at time of sending message...
// Send text prompt message
llmHelper.appendToMessages({ role: "user", content: 'Hello OpenAI!' });

​
Handling Events

The transport implements the various RTVI event handlers. Check out the docs or samples for more info.

​
More Information

Demo

OpenAI Realtime Basic Demo

Source

OpenAIRealTimeWebRTCTransport

Package

@pipecat-ai/openai-realtime-webrtc-transport

GeminiLiveWebSocketTransportSDK Introduction