The Pipecat JavaScript client listens for messages and events from the bot via the transport layer. This allows you to respond to changes in state, errors, and other events. The client implements the RTVI standard for these communications.
The bot has been instantiated, its pipeline is configured, and it is receiving user media and interactions. This method is passed a BotReadyData object, which contains the RTVI version number. Since the bot is remote and may be using a different version of RTVI than the client, you can use the passed version string to check for compatibility.
Bot connected to the transport and is configuring. Note: bot connectivity does not infer that its pipeline is yet ready to run. Please use onBotReady instead.
Receives custom messages sent from the server to the client. This provides a generic channel for server-to-client communication. The data structure is flexible and defined by the server implementation.
Error signalled by the bot. This could be due to a malformed config update or an unknown action dispatch or the ability to complete the requested action.
Lists available local media microphone devices. Triggered when a new device becomes available, a device is removed, or in response to pcClient.initDevices().
Lists available local media camera devices. Triggered when a new device becomes available, a device is removed, or in response to pcClient.initDevices().
Remote audio gain level (0 to 1). Note: if more than one participant is connected to the transport, the participant property details the associated peer/bot.
The local user started speaking. This method is more reliable than using audio
gain and is the result of the bot’s VAD (voice activity detection) model. This
provides a more accurate result in noisy environments.
Bot LLM search response text generated by the LLM service. This is typically used for search or retrieval tasks.
Search capabilities are currently only supported by Google Gemini. To take advantage of this event, your pipeline must include a GoogleLLMService and your pipeline task should include the GoogleRTVIObserver in lieu of the typical RTVIObserver.
If your TTS service supports streamed responses over sockets, the text parameter contains the words from TTS service as they are spoken. If you are using a HTTP based TTS service, the text parameter will contain the full text of the TTS response.
Each callback described above has a corresponding event that can be listened for using the .on() method. This allows you to handle the same functionality using either callbacks or event listeners, depending on your preferred architecture.Here’s the complete reference mapping events to their corresponding callbacks: