Callbacks and Events

The Pipecat JavaScript client listens for messages and events from the bot via the transport layer. This allows you to respond to changes in state, errors, and other events. The client implements the RTVI standard for these communications.

Event Handling Options

You can handle events in two ways:

1. Callbacks

Define handlers in the client constructor:

const pcClient = new PipecatClient({
  callbacks: {
    onBotReady: () => console.log("Bot ready via callback"),
    // ... other callbacks
  },
});

2. Event Listeners

Add handlers using the event emitter pattern:

pcClient.on(RTVIEvent.BotReady, () => console.log("Bot ready via event"));

Events and callbacks provide the same functionality. Choose the pattern that best fits your application’s architecture.

Callbacks

State and connectivity

Local user successfully established a connection to the transport.

Local user disconnected from the transport, either intentionally by calling pcClient.disconnect() or due to an error.

state:TransportState

Provides a TransportState string representing the connectivity state of the local client. See transports for state explanation.

botResponse: unknown

A call to startBot() (i.e. a pre-connection REST endpoint) was successful and the bot should now be started or in the process of starting. The callback receives any data returned from your endpoint.

botReadyData:BotReadyData

The bot has been instantiated, its pipeline is configured, and it is receiving user media and interactions. This method is passed a BotReadyData object, which contains the RTVI version number. Since the bot is remote and may be using a different version of RTVI than the client, you can use the passed version string to check for compatibility.

Bot connected to the transport and is configuring. Note: bot connectivity does not infer that its pipeline is yet ready to run. Please use onBotReady instead.

participant: Participant

Bot disconnected from the transport. This may occur due to session expiry, a pipeline error or for any reason the server deems the session over. By default, the client will also disconnect when this fires. Set disconnectOnBotDisconnect: false in the constructor to keep the client connected.

participant: Participant

A participant joined the session. Fires for all participants, including the bot and the local user.

participant: Participant

A participant left the session. Fires for all participants, including the bot and the local user.

Messages and errors

data:any

Receives custom messages sent from the server to the client. This provides a generic channel for server-to-client communication. The data structure is flexible and defined by the server implementation.

message:RTVIMessage

Response error when an action fails or an unknown message type is sent from the client.

message:RTVIMessage

Error signalled by the bot. This could be due to a malformed config update or an unknown action dispatch or the inability to complete a client request. The message parameter is of type error and matches the RTVI standard. Its data field includes a message string that describes the error and a fatal boolean indicating if the error is unrecoverable and resulted in a bot disconnection. If fatal is true, the client will automatically disconnect.

Media and devices

mics:MediaDeviceInfo[]

Lists available local media microphone devices. Triggered when a new device becomes available, a device is removed, or in response to pcClient.initDevices().

cams:MediaDeviceInfo[]

Lists available local media camera devices. Triggered when a new device becomes available, a device is removed, or in response to pcClient.initDevices().

speakers:MediaDeviceInfo[]

Lists available local speaker devices. Triggered when a new device becomes available, a device is removed, or in response to pcClient.initDevices().

mic:MediaDeviceInfo

User selected a new microphone as their selected/active device.

cam:MediaDeviceInfo

User selected a new camera as their selected/active device.

speaker:MediaDeviceInfo

User selected a new speaker as their selected/active device.

error:DeviceError

Error related to media devices, such as camera or microphone issues. This could be due to permissions, device unavailability, or other related problems. See the DeviceError section for more details about the return type.

track: MediaStreamTrack, participant:Participant

Media track from a local or remote participant/bot was started and playable. Can be either an audio or video track.

track: MediaStreamTrack, participant:Participant

Media track from a local or remote participant/bot was stopped and no longer playable.

track: MediaStreamTrack, participant:Participant

Media track from a local or remote participant’s screenshare was started and playable. Can be either an audio or video track.

track: MediaStreamTrack, participant:Participant

Media track from a local or remote participant’s screenshare was stopped and no longer playable.

Audio and Voice Activity

level:number

Local audio gain level (0 to 1).

level: number, participant: Participant

Remote audio gain level (0 to 1). Note: if more than one participant is connected to the transport, the participant property details the associated peer/bot.

The bot started speaking/sending speech audio.

The bot stopped speaking/sending speech audio.

The local user started speaking. This method is more reliable than using audio gain and is the result of the bot’s VAD (voice activity detection) model. This provides a more accurate result in noisy environments.

The local user stopped speaking, indicated by the VAD model.

The server has started ignoring audio from the client (server-side muting). The client should continue sending audio normally but may want to show an indication to the user that their input is not being processed. See User Input Muting for more details.

The server has stopped ignoring audio from the client (server-side muting ended). The client can update its UI to indicate that the user’s input is being processed again.

Transcription

TranscriptData

Transcribed local user input (both partial and final).Callback receives a TranscriptData object:

Show TranscriptData

string

The transcribed text.

boolean

Indicates if the text is final (true) or partial (false).

string

The timestamp of the transcription.

string

The ID of the user the transcription is for.

BotOutputData

A best-effort stream of the bot’s output text, including both spoken and unspoken content. This callback is triggered as the bot aggregates the LLM’s response into sentences or other logical text blocks as well as word-by-word during TTS synthesis. The callback receives a BotOutputData object:

Show BotOutputData

string

The aggregated output text from the bot.

boolean

Indicates if the text has been spoken by the bot.

'sentence' | 'word' | string

The aggregation type used for this output (e.g., “sentence”, “code”). “sentence” and “word” are reserved aggregation types defined by the RTVI standard. Other aggregation types may be defined by custom text aggregators used by the server. “word” aggregated outputs are sent at the time of TTS synthesis for real-time word-level streaming and can be used in lieu of onBotTtsText if desired.

text:BotLLMTextData

DEPRECATED in favor of onBotOutput in Pipecat version 0.0.95 and client-js version 1.5.0

Finalized bot output text generated by the LLM. Sentence aggregated.

Service-specific Events

BotLLMSearchResponseData

Bot LLM search response text generated by the LLM service. This is typically used for search or retrieval tasks.

Search capabilities are currently only supported by Google Gemini. To take advantage of this event, your pipeline must include a GoogleLLMService and your pipeline worker should include the GoogleRTVIObserver in lieu of the typical RTVIObserver.

Show BotLLMSearchResponseData

string

The search result text.

string

The rendered content of the search result.

LLMSearchOrigin[]

The origins of the search result.

Show LLMSearchOrigin

string

The URI of the site where the search result was found.

string

The title of the site where the search result was found.

LLMSearchResult[]

The individual search results.

Show LLMSearchResult

string

The text of the search result.

number[]

The confidence scores for the search result.

BotLLMTextData

Streamed LLM token response text generated by the LLM service.

Show BotLLMTextData

string

The text of the LLM response.

LLM service inference started.

LLM service inference concluded.

BotTTSTextData

If your TTS service supports streamed responses over sockets, the text parameter contains the words from TTS service as they are spoken. If you are using a HTTP based TTS service, the text parameter will contain the full text of the TTS response.

Hide BotTTSTextData

string

The text of the LLM response.

TTS service started inference.

TTS service inference concluded.

Function Calling

LLMFunctionCallStartedData

A function call has been initiated by the LLM. The metadata included depends on the server’s function_call_report_level configuration.

Show LLMFunctionCallStartedData

string

Name of the function being called. Only included if report level is NAME or FULL.

LLMFunctionCallInProgressData

A function call is in progress. This replaces the deprecated onLLMFunctionCall callback and is the event that triggers registered FunctionCallHandlers when a function_name is present.

Show LLMFunctionCallInProgressData

string

Name of the function being called. Only included if report level is NAME or FULL.

string

Unique identifier for this function call.

Record<string, unknown>

Arguments passed to the function. Only included if report level is FULL.

LLMFunctionCallStoppedData

A function call has completed or been cancelled.

Show LLMFunctionCallStoppedData

string

Name of the function that was called. Only included if report level is NAME or FULL.

string

Identifier matching the original function call.

boolean

Whether the function call was cancelled before completing.

unknown

The result of the function call, if available. Only included if report level is FULL.

LLMFunctionCallData

DEPRECATED in favor of onLLMFunctionCallInProgress in Pipecat version 0.0.102 and client-js version 1.6.0

A function call request from the LLM.

User interface

data:UICommandData

The bot sent a UI command for your app to act on. data is { command, payload }; standard commands are scroll_to, highlight, select_text, click, focus, set_input_value, toast, and navigate (the React SDK ships default handlers). See the RTVI standard for payload shapes.

data:UIJobGroupData

A lifecycle envelope for background work the bot dispatched, for rendering a progress card. data.kind is one of group_started, job_update, job_completed, group_completed. Cancel a group with cancelUIJobGroup().

Other

default:"data:PipecatMetricsData"

Pipeline mterics data provided by Pipecat. Learn more.

Events

Each callback described above has a corresponding event that can be listened for using the .on() method. This allows you to handle the same functionality using either callbacks or event listeners, depending on your preferred architecture. Here’s the complete reference mapping events to their corresponding callbacks:

State and connectivity Events

Event Name	Callback Name	Data Type
`Connected`	`onConnected`	-
`Disconnected`	`onDisconnected`	-
`TransportStateChanged`	`onTransportStateChanged`	`TransportState`
`BotReady`	`onBotReady`	`BotReadyData`
`BotConnected`	`onBotConnected`	-
`BotDisconnected`	`onBotDisconnected`	`Participant`
`ParticipantConnected`	`onParticipantJoined`	`Participant`
`ParticipantLeft`	`onParticipantLeft`	`Participant`

Message and Error Events

Event Name	Callback Name	Data Type
`ServerMessage`	`onServerMessage`	`any`
`MessageError`	`onMessageError`	`RTVIMessage`
`Error`	`onError`	`RTVIMessage`

Media Events

Event Name	Callback Name	Data Type
`TrackStarted`	`onTrackStarted`	`MediaStreamTrack, Participant`
`TrackStopped`	`onTrackStopped`	`MediaStreamTrack, Participant`
`AvailableMicsUpdated`	`onAvailableMicsUpdated`	`MediaDeviceInfo[]`
`AvailableCamsUpdated`	`onAvailableCamsUpdated`	`MediaDeviceInfo[]`
`MicUpdated`	`onMicUpdated`	`MediaDeviceInfo`
`CamUpdated`	`onCamUpdated`	`MediaDeviceInfo`
`SpeakerUpdated`	`onSpeakerUpdated`	`MediaDeviceInfo`
`DeviceError`	`onDeviceError`	`DeviceError`

Audio Activity Events

Event Name	Callback Name	Data Type
`LocalAudioLevel`	`onLocalAudioLevel`	`number`
`RemoteAudioLevel`	`onRemoteAudioLevel`	`number, Participant`
`BotStartedSpeaking`	`onBotStartedSpeaking`	-
`BotStoppedSpeaking`	`onBotStoppedSpeaking`	-
`UserStartedSpeaking`	`onUserStartedSpeaking`	-
`UserStoppedSpeaking`	`onUserStoppedSpeaking`	-
`UserMuteStarted`	`onUserMuteStarted`	-
`UserMuteStopped`	`onUserMuteStopped`	-

Text and Transcription Events

Event Name	Callback Name	Data Type
`UserTranscript`	`onUserTranscript`	`TranscriptData`
`BotOutput`	`onBotOutput`	`BotOutputData`
~~`BotTranscript`~~	~~`onBotTranscript`~~	`BotLLMTextData`
`BotLlmText`	`onBotLlmText`	`BotLLMTextData`
`BotTtsText`	`onBotTtsText`	`BotTTSTextData`

Service State Events

Event Name	Callback Name	Data Type
`BotLlmSearchResponse`	`onBotLlmSearchResponse`	`BotLLMSearchResponseData`
`BotLlmStarted`	`onBotLlmStarted`	-
`BotLlmStopped`	`onBotLlmStopped`	-
`BotTtsStarted`	`onBotTtsStarted`	-
`BotTtsStopped`	`onBotTtsStopped`	-

Function Call Events

Event Name	Callback Name	Data Type
`LLMFunctionCallStarted`	`onLLMFunctionCallStarted`	`LLMFunctionCallStartedData`
`LLMFunctionCallInProgress`	`onLLMFunctionCallInProgress`	`LLMFunctionCallInProgressData`
`LLMFunctionCallStopped`	`onLLMFunctionCallStopped`	`LLMFunctionCallStoppedData`
~~`LLMFunctionCall`~~	~~`onLLMFunctionCall`~~	`LLMFunctionCallData`

User Interface Events

Event Name	Callback Name	Data Type
`UICommand`	`onUICommand`	`UICommandData`
`UIJobGroup`	`onUIJobGroup`	`UIJobGroupData`

Streaming snapshots and sending UI events are client methods; see the RTVI standard for payload types.

Other Events

Event Name	Callback Name	Data Type
`Metrics`	`onMetrics`	`PipecatMetricsData`

Usage Example

import { PipecatClient, RTVIEvent } from "@pipecat-ai/client-js";

// Using callbacks
const pcClient = new PipecatClient({
  callbacks: {
    onBotReady: () => console.log("Bot ready via callback"),
    onUserTranscript: (data) => console.log("Transcript:", data.text),
  },
});

// Alternate approach: Using event listeners
pcClient.on(RTVIEvent.BotReady, () => {
  console.log("Bot ready via event");
});

Pipecat Server

Client SDKs

Pipecat Flows

Pipecat Cloud

CLI

Pipecat Context Hub

Callbacks and Events

Event Handling Options

1. Callbacks

2. Event Listeners

Callbacks

State and connectivity

Messages and errors

Media and devices

Audio and Voice Activity

Transcription

Service-specific Events

Function Calling

User interface

Other

Events

State and connectivity Events

Message and Error Events

Media Events

Audio Activity Events

Text and Transcription Events

Service State Events

Function Call Events

User Interface Events

Other Events

Usage Example

Transport Compatibility

​Event Handling Options

​1. Callbacks

​2. Event Listeners

​Callbacks

​State and connectivity

​Messages and errors

​Media and devices

​Audio and Voice Activity

​Transcription

​Service-specific Events

​Function Calling

​User interface

​Other

​Events

​State and connectivity Events

​Message and Error Events

​Media Events

​Audio Activity Events

​Text and Transcription Events

​Service State Events

​Function Call Events

​User Interface Events

​Other Events

​Usage Example

​Transport Compatibility

Event Handling Options

1. Callbacks

2. Event Listeners

Callbacks

State and connectivity

Messages and errors

Media and devices

Audio and Voice Activity

Transcription

Service-specific Events

Function Calling

User interface

Other

Events

State and connectivity Events

Message and Error Events

Media Events

Audio Activity Events

Text and Transcription Events

Service State Events

Function Call Events

User Interface Events

Other Events

Usage Example

Transport Compatibility