Callbacks and events

The Pipecat JavaScript client listens for messages and events from the bot via the transport layer. This allows you to respond to changes in state, errors, and other events. The client implements the RTVI standard for these communications.

Event Handling Options

You can handle events in two ways:

1. Callbacks

Define handlers in the client constructor:

const pcClient = new PipecatClient({
  callbacks: {
    onBotReady: () => console.log("Bot ready via callback"),
    // ... other callbacks
  },
});

2. Event Listeners

Add handlers using the event emitter pattern:

pcClient.on(RTVIEvent.BotReady, () => console.log("Bot ready via event"));

Events and callbacks provide the same functionality. Choose the pattern that best fits your application’s architecture.

Callbacks

State and connectivity

onConnected

Local user successfully established a connection to the transport.

onDisconnected

Local user disconnected from the transport, either intentionally by calling pcClient.disconnect() or due to an error.

onTransportStateChanged

state:TransportState

Provides a TransportState string representing the connectivity state of the local client. See transports for state explanation.

onBotReady

botReadyData:BotReadyData

The bot has been instantiated, its pipeline is configured, and it is receiving user media and interactions. This method is passed a BotReadyData object, which contains the RTVI version number. Since the bot is remote and may be using a different version of RTVI than the client, you can use the passed version string to check for compatibility.

onBotConnected

Bot connected to the transport and is configuring. Note: bot connectivity does not infer that its pipeline is yet ready to run. Please use onBotReady instead.

onBotDisconnected

participant: Participant

Bot disconnected from the transport. This may occur due to session expiry, a pipeline error or for any reason the server deems the session over.

onParticipantJoined

participant: Participant

A non-bot participant joined the session.

onParticipantLeft

participant: Participant

A non-bot participant left the session. Note: excluded local participant.

Messages and errors

onServerMessage

data:any

Receives custom messages sent from the server to the client. This provides a generic channel for server-to-client communication. The data structure is flexible and defined by the server implementation.

onMessageError

message:RTVIMessage

Response error when an action fails or an unknown message type is sent from the client.

onError

message:RTVIMessage

Error signalled by the bot. This could be due to a malformed config update or an unknown action dispatch or the ability to complete the requested action.

Media and devices

onAvailableMicsUpdated

mics:MediaDeviceInfo[]

Lists available local media microphone devices. Triggered when a new device becomes available, a device is removed, or in response to pcClient.initDevices().

onAvailableCamsUpdated

cams:MediaDeviceInfo[]

Lists available local media camera devices. Triggered when a new device becomes available, a device is removed, or in response to pcClient.initDevices().

onAvailableSpeakersUpdated

speakers:MediaDeviceInfo[]

Lists available local speaker devices. Triggered when a new device becomes available, a device is removed, or in response to pcClient.initDevices().

onMicUpdated

mic:MediaDeviceInfo

User selected a new microphone as their selected/active device.

onCamUpdated

cam:MediaDeviceInfo

User selected a new camera as their selected/active device.

onSpeakerUpdated

speaker:MediaDeviceInfo

User selected a new speaker as their selected/active device.

onTrackStarted

track: MediaStreamTrack, participant:Participant

Media track from a local or remote participant/bot was started and playable. Can be either an audio or video track.

onTrackStopped

track: MediaStreamTrack, participant:Participant

Media track from a local or remote participant/bot was stopped and no longer playable.

onScreenTrackStarted

track: MediaStreamTrack, participant:Participant

Media track from a local or remote participant’s screenshare was started and playable. Can be either an audio or video track.

onScreenTrackStopped

track: MediaStreamTrack, participant:Participant

Media track from a local or remote participant’s screenshare was stopped and no longer playable.

Audio and Voice Activity

onLocalAudioLevel

level:number

Local audio gain level (0 to 1).

onRemoteAudioLevel

level: number, participant: Participant

Remote audio gain level (0 to 1). Note: if more than one participant is connected to the transport, the participant property details the associated peer/bot.

onBotStartedSpeaking

The bot started speaking/sending speech audio.

onBotStoppedSpeaking

The bot stopped speaking/sending speech audio.

onUserStartedSpeaking

The local user started speaking. This method is more reliable than using audio gain and is the result of the bot’s VAD (voice activity detection) model. This provides a more accurate result in noisy environments.

onUserStoppedSpeaking

The local user stopped speaking, indicated by the VAD model.

Transcription

onUserTranscript

TranscriptData

Transcribed local user input (both partial and final).Callback receives a TranscriptData object:

Show TranscriptData

text

string

The transcribed text.

final

boolean

Indicates if the text is final (true) or partial (false).

timestamp

string

The timestamp of the transcription.

user_id

string

The ID of the user the transcription is for.

onBotTranscript

text:BotLLMTextData

Finalized bot output text generated by the LLM. Sentence aggregated.

Service-specific Events

onBotLlmSearchResponse

BotLLMSearchResponseData

Bot LLM search response text generated by the LLM service. This is typically used for search or retrieval tasks.

Search capabilities are currently only supported by Google Gemini. To take advantage of this event, your pipeline must include a GoogleLLMService and your pipeline task should include the GoogleRTVIObserver in lieu of the typical RTVIObserver.

Show BotLLMSearchResponseData

search_result

string

The search result text.

rendered_content

string

The rendered content of the search result.

origins

LLMSearchOrigin[]

The origins of the search result.

Show LLMSearchOrigin

site_uri

string

The URI of the site where the search result was found.

site_title

string

The title of the site where the search result was found.

results

LLMSearchResult[]

The individual search results.

Show LLMSearchResult

text

string

The text of the search result.

confidence

number[]

The confidence scores for the search result.

onBotLlmText

BotLLMTextData

Streamed LLM token response text generated by the LLM service.

Show BotLLMTextData

text

string

The text of the LLM response.

onBotLlmStarted

LLM service inference started.

onBotLlmStopped

LLM service inference concluded.

onBotTtsText

BotTTSTextData

If your TTS service supports streamed responses over sockets, the text parameter contains the words from TTS service as they are spoken. If you are using a HTTP based TTS service, the text parameter will contain the full text of the TTS response.

Hide BotTTSTextData

text

string

The text of the LLM response.

onBotTtsStarted

TTS service started inference.

onBotTtsStopped

TTS service inference concluded.

Other

onMetrics

default:"data:PipecatMetricsData"

Pipeline data provided by Pipecat. Please see Pipecat documentation for more information.

Events

Each callback described above has a corresponding event that can be listened for using the .on() method. This allows you to handle the same functionality using either callbacks or event listeners, depending on your preferred architecture. Here’s the complete reference mapping events to their corresponding callbacks:

State and connectivity Events

Event Name	Callback Name	Data Type
`Connected`	`onConnected`	-
`Disconnected`	`onDisconnected`	-
`TransportStateChanged`	`onTransportStateChanged`	`TransportState`
`BotReady`	`onBotReady`	`BotReadyData`
`BotConnected`	`onBotConnected`	-
`BotDisconnected`	`onBotDisconnected`	`Participant`
`ParticipantConnected`	`onParticipantJoined`	`Participant`
`ParticipantLeft`	`onParticipantLeft`	`Participant`

Message and Error Events

Event Name	Callback Name	Data Type
`ServerMessage`	`onServerMessage`	`any`
`MessageError`	`onMessageError`	`RTVIMessage`
`Error`	`onError`	`RTVIMessage`

Media Events

Event Name	Callback Name	Data Type
`TrackStarted`	`onTrackStarted`	`MediaStreamTrack, Participant`
`TrackStopped`	`onTrackStopped`	`MediaStreamTrack, Participant`
`AvailableMicsUpdated`	`onAvailableMicsUpdated`	`MediaDeviceInfo[]`
`AvailableCamsUpdated`	`onAvailableCamsUpdated`	`MediaDeviceInfo[]`
`MicUpdated`	`onMicUpdated`	`MediaDeviceInfo`
`CamUpdated`	`onCamUpdated`	`MediaDeviceInfo`

Audio Activity Events

Event Name	Callback Name	Data Type
`LocalAudioLevel`	`onLocalAudioLevel`	`number`
`RemoteAudioLevel`	`onRemoteAudioLevel`	`number, Participant`
`BotStartedSpeaking`	`onBotStartedSpeaking`	-
`BotStoppedSpeaking`	`onBotStoppedSpeaking`	-
`UserStartedSpeaking`	`onUserStartedSpeaking`	-
`UserStoppedSpeaking`	`onUserStoppedSpeaking`	-

Text and Transcription Events

Event Name	Callback Name	Data Type
`UserTranscript`	`onUserTranscript`	`TranscriptData`
`BotTranscript`	`onBotTranscript`	`BotLLMTextData`
`BotLlmText`	`onBotLlmText`	`BotLLMTextData`
`BotTtsText`	`onBotTtsText`	`BotTTSTextData`

Service State Events

Event Name	Callback Name	Data Type
`BotLlmSearchResponse`	`onBotLlmSearchResponse`	`BotLLMSearchResponseData`
`BotLlmStarted`	`onBotLlmStarted`	-
`BotLlmStopped`	`onBotLlmStopped`	-
`BotTtsStarted`	`onBotTtsStarted`	-
`BotTtsStopped`	`onBotTtsStopped`	-

Other Events

Event Name	Callback Name	Data Type
`Metrics`	`onMetrics`	`PipecatMetricsData`

Usage Example

import { PipecatClient, RTVIEvent } from "@pipecat-ai/client-js";

// Using callbacks
const pcClient = new PipecatClient({
  callbacks: {
    onBotReady: () => console.log("Bot ready via callback"),
    onUserTranscript: (data) => console.log("Transcript:", data.text),
  },
});

// Alternate approach: Using event listeners
pcClient.on(RTVIEvent.BotReady, () => {
  console.log("Bot ready via event");
});

Javascript SDK

React SDK

React Native SDK

iOS SDK

Android SDK

C++ SDK

Callbacks and events

Event Handling Options

1. Callbacks

2. Event Listeners

Callbacks

State and connectivity

Messages and errors

Media and devices

Audio and Voice Activity

Transcription

Service-specific Events

Other

Events

State and connectivity Events

Message and Error Events

Media Events

Audio Activity Events

Text and Transcription Events

Service State Events

Other Events

Usage Example

Javascript SDK

React SDK

React Native SDK

iOS SDK

Android SDK

C++ SDK

​Event Handling Options

​1. Callbacks

​2. Event Listeners

​Callbacks

​State and connectivity

​Messages and errors

​Media and devices

​Audio and Voice Activity

​Transcription

​Service-specific Events

​Other

​Events

​State and connectivity Events

​Message and Error Events

​Media Events

​Audio Activity Events

​Text and Transcription Events

​Service State Events

​Other Events

​Usage Example

Event Handling Options

1. Callbacks

2. Event Listeners

Callbacks

State and connectivity

Messages and errors

Media and devices

Audio and Voice Activity

Transcription

Service-specific Events

Other

Events

State and connectivity Events

Message and Error Events

Media Events

Audio Activity Events

Text and Transcription Events

Service State Events

Other Events

Usage Example