The Pipecat JavaScript client listens for messages and events from the bot via the transport layer. This allows you to respond to changes in state, errors, and other events. The client implements the RTVI standard for these communications.

Event Handling Options

You can handle events in two ways:

1. Callbacks

Define handlers in the client constructor:
const pcClient = new PipecatClient({
  callbacks: {
    onBotReady: () => console.log("Bot ready via callback"),
    // ... other callbacks
  },
});

2. Event Listeners

Add handlers using the event emitter pattern:
pcClient.on(RTVIEvent.BotReady, () => console.log("Bot ready via event"));
Events and callbacks provide the same functionality. Choose the pattern that best fits your application’s architecture.

Callbacks

State and connectivity

onConnected
Local user successfully established a connection to the transport.
onDisconnected
Local user disconnected from the transport, either intentionally by calling pcClient.disconnect() or due to an error.
onTransportStateChanged
state:TransportState
Provides a TransportState string representing the connectivity state of the local client. See transports for state explanation.
onBotReady
botReadyData:BotReadyData
The bot has been instantiated, its pipeline is configured, and it is receiving user media and interactions. This method is passed a BotReadyData object, which contains the RTVI version number. Since the bot is remote and may be using a different version of RTVI than the client, you can use the passed version string to check for compatibility.
onBotConnected
Bot connected to the transport and is configuring. Note: bot connectivity does not infer that its pipeline is yet ready to run. Please use onBotReady instead.
onBotDisconnected
participant: Participant
Bot disconnected from the transport. This may occur due to session expiry, a pipeline error or for any reason the server deems the session over.
onParticipantJoined
participant: Participant
A non-bot participant joined the session.
onParticipantLeft
participant: Participant
A non-bot participant left the session. Note: excluded local participant.

Messages and errors

onServerMessage
data:any
Receives custom messages sent from the server to the client. This provides a generic channel for server-to-client communication. The data structure is flexible and defined by the server implementation.
onMessageError
message:RTVIMessage
Response error when an action fails or an unknown message type is sent from the client.
onError
message:RTVIMessage
Error signalled by the bot. This could be due to a malformed config update or an unknown action dispatch or the ability to complete the requested action.

Media and devices

onAvailableMicsUpdated
mics:MediaDeviceInfo[]
Lists available local media microphone devices. Triggered when a new device becomes available, a device is removed, or in response to pcClient.initDevices().
onAvailableCamsUpdated
cams:MediaDeviceInfo[]
Lists available local media camera devices. Triggered when a new device becomes available, a device is removed, or in response to pcClient.initDevices().
onAvailableSpeakersUpdated
speakers:MediaDeviceInfo[]
Lists available local speaker devices. Triggered when a new device becomes available, a device is removed, or in response to pcClient.initDevices().
onMicUpdated
mic:MediaDeviceInfo
User selected a new microphone as their selected/active device.
onCamUpdated
cam:MediaDeviceInfo
User selected a new camera as their selected/active device.
onSpeakerUpdated
speaker:MediaDeviceInfo
User selected a new speaker as their selected/active device.
onTrackStarted
track: MediaStreamTrack, participant:Participant
Media track from a local or remote participant/bot was started and playable. Can be either an audio or video track.
onTrackStopped
track: MediaStreamTrack, participant:Participant
Media track from a local or remote participant/bot was stopped and no longer playable.
onScreenTrackStarted
track: MediaStreamTrack, participant:Participant
Media track from a local or remote participant’s screenshare was started and playable. Can be either an audio or video track.
onScreenTrackStopped
track: MediaStreamTrack, participant:Participant
Media track from a local or remote participant’s screenshare was stopped and no longer playable.

Audio and Voice Activity

onLocalAudioLevel
level:number
Local audio gain level (0 to 1).
onRemoteAudioLevel
level: number, participant: Participant
Remote audio gain level (0 to 1). Note: if more than one participant is connected to the transport, the participant property details the associated peer/bot.
onBotStartedSpeaking
The bot started speaking/sending speech audio.
onBotStoppedSpeaking
The bot stopped speaking/sending speech audio.
onUserStartedSpeaking
The local user started speaking. This method is more reliable than using audio gain and is the result of the bot’s VAD (voice activity detection) model. This provides a more accurate result in noisy environments.
onUserStoppedSpeaking
The local user stopped speaking, indicated by the VAD model.

Transcription

onUserTranscript
TranscriptData
Transcribed local user input (both partial and final).Callback receives a TranscriptData object:
onBotTranscript
text:BotLLMTextData
Finalized bot output text generated by the LLM. Sentence aggregated.

Service-specific Events

onBotLlmSearchResponse
BotLLMSearchResponseData
Bot LLM search response text generated by the LLM service. This is typically used for search or retrieval tasks.
Search capabilities are currently only supported by Google Gemini. To take advantage of this event, your pipeline must include a GoogleLLMService and your pipeline task should include the GoogleRTVIObserver in lieu of the typical RTVIObserver.
onBotLlmText
BotLLMTextData
Streamed LLM token response text generated by the LLM service.
onBotLlmStarted
LLM service inference started.
onBotLlmStopped
LLM service inference concluded.
onBotTtsText
BotTTSTextData
If your TTS service supports streamed responses over sockets, the text parameter contains the words from TTS service as they are spoken. If you are using a HTTP based TTS service, the text parameter will contain the full text of the TTS response.
onBotTtsStarted
TTS service started inference.
onBotTtsStopped
TTS service inference concluded.

Other

onMetrics
default:"data:PipecatMetricsData"
Pipeline data provided by Pipecat. Please see Pipecat documentation for more information.

Events

Each callback described above has a corresponding event that can be listened for using the .on() method. This allows you to handle the same functionality using either callbacks or event listeners, depending on your preferred architecture. Here’s the complete reference mapping events to their corresponding callbacks:

State and connectivity Events

Event NameCallback NameData Type
ConnectedonConnected-
DisconnectedonDisconnected-
TransportStateChangedonTransportStateChangedTransportState
BotReadyonBotReadyBotReadyData
BotConnectedonBotConnected-
BotDisconnectedonBotDisconnectedParticipant
ParticipantConnectedonParticipantJoinedParticipant
ParticipantLeftonParticipantLeftParticipant

Message and Error Events

Event NameCallback NameData Type
ServerMessageonServerMessageany
MessageErroronMessageErrorRTVIMessage
ErroronErrorRTVIMessage

Media Events

Event NameCallback NameData Type
TrackStartedonTrackStartedMediaStreamTrack, Participant
TrackStoppedonTrackStoppedMediaStreamTrack, Participant
AvailableMicsUpdatedonAvailableMicsUpdatedMediaDeviceInfo[]
AvailableCamsUpdatedonAvailableCamsUpdatedMediaDeviceInfo[]
MicUpdatedonMicUpdatedMediaDeviceInfo
CamUpdatedonCamUpdatedMediaDeviceInfo

Audio Activity Events

Event NameCallback NameData Type
LocalAudioLevelonLocalAudioLevelnumber
RemoteAudioLevelonRemoteAudioLevelnumber, Participant
BotStartedSpeakingonBotStartedSpeaking-
BotStoppedSpeakingonBotStoppedSpeaking-
UserStartedSpeakingonUserStartedSpeaking-
UserStoppedSpeakingonUserStoppedSpeaking-

Text and Transcription Events

Event NameCallback NameData Type
UserTranscriptonUserTranscriptTranscriptData
BotTranscriptonBotTranscriptBotLLMTextData
BotLlmTextonBotLlmTextBotLLMTextData
BotTtsTextonBotTtsTextBotTTSTextData

Service State Events

Event NameCallback NameData Type
BotLlmSearchResponseonBotLlmSearchResponseBotLLMSearchResponseData
BotLlmStartedonBotLlmStarted-
BotLlmStoppedonBotLlmStopped-
BotTtsStartedonBotTtsStarted-
BotTtsStoppedonBotTtsStopped-

Other Events

Event NameCallback NameData Type
MetricsonMetricsPipecatMetricsData

Usage Example

import { PipecatClient, RTVIEvent } from "@pipecat-ai/client-js";

// Using callbacks
const pcClient = new PipecatClient({
  callbacks: {
    onBotReady: () => console.log("Bot ready via callback"),
    onUserTranscript: (data) => console.log("Transcript:", data.text),
  },
});

// Alternate approach: Using event listeners
pcClient.on(RTVIEvent.BotReady, () => {
  console.log("Bot ready via event");
});