Hooks - Pipecat

The Pipecat React SDK provides hooks for accessing client functionality, managing media devices, and handling events.

usePipecatClient

Provides access to the PipecatClient instance originally passed to PipecatClientProvider.

import { usePipecatClient } from "@pipecat-ai/client-react";

function MyComponent() {
  const pcClient = usePipecatClient();

  await pcClient.startBotAndConnect({
    endpoint: '/api/start',
    requestData: {
      // Any custom data your /start endpoint requires
    }
  });
}

useRTVIClientEvent

Allows subscribing to RTVI client events. It is advised to wrap handlers with useCallback.

import { useCallback } from "react";
import { RTVIEvent, TransportState } from "@pipecat-ai/client-js";
import { useRTVIClientEvent } from "@pipecat-ai/client-react";

function EventListener() {
  useRTVIClientEvent(
    RTVIEvent.TransportStateChanged,
    useCallback((transportState: TransportState) => {
      console.log("Transport state changed to", transportState);
    }, [])
  );
}

Arguments

event

RTVIEvent

required

handler

function

required

usePipecatClientMediaDevices

Manage and list available media devices.

import { usePipecatClientMediaDevices } from "@pipecat-ai/client-react";

function DeviceSelector() {
  const {
    availableCams,
    availableMics,
    selectedCam,
    selectedMic,
    updateCam,
    updateMic,
  } = usePipecatClientMediaDevices();

  return (
    <>
      <select
        name="cam"
        onChange={(ev) => updateCam(ev.target.value)}
        value={selectedCam?.deviceId}
      >
        {availableCams.map((cam) => (
          <option key={cam.deviceId} value={cam.deviceId}>
            {cam.label}
          </option>
        ))}
      </select>
      <select
        name="mic"
        onChange={(ev) => updateMic(ev.target.value)}
        value={selectedMic?.deviceId}
      >
        {availableMics.map((mic) => (
          <option key={mic.deviceId} value={mic.deviceId}>
            {mic.label}
          </option>
        ))}
      </select>
    </>
  );
}

usePipecatClientMediaTrack

Access audio and video tracks.

import { usePipecatClientMediaTrack } from "@pipecat-ai/client-react";

function MyTracks() {
  const localAudioTrack = usePipecatClientMediaTrack("audio", "local");
  const botAudioTrack = usePipecatClientMediaTrack("audio", "bot");
}

Arguments

trackType

'audio' | 'video'

required

participantType

'bot' | 'local'

required

usePipecatClientTransportState

Returns the current transport state.

import { usePipecatClientTransportState } from "@pipecat-ai/client-react";

function ConnectionStatus() {
  const transportState = usePipecatClientTransportState();
}

usePipecatClientCamControl

Controls the local participant’s camera state.

import { usePipecatClientCamControl } from "@pipecat-ai/client-react";
function CamToggle() {
  const { enableCam, isCamEnabled } = usePipecatClientCamControl();

  return (
    <button onClick={() => enableCam(!isCamEnabled)}>
      {isCamEnabled ? "Disable Camera" : "Enable Camera"}
    </button>
  );
}

usePipecatClientMicControl

Controls the local participant’s microphone state.

import { usePipecatClientMicControl } from "@pipecat-ai/client-react";
function MicToggle() {
  const { enableMic, isMicEnabled } = usePipecatClientMicControl();

  return (
    <button onClick={() => enableMic(!isMicEnabled)}>
      {isMicEnabled ? "Disable Microphone" : "Enable Microphone"}
    </button>
  );
}

usePipecatConversation

The primary hook for accessing the conversation message stream. Returns the current list of messages (ordered for display) and a function to inject messages programmatically. Each assistant message’s text parts are split into spoken and unspoken segments based on real-time speech progress, so you can style them differently (e.g. dim unspoken text).

import { usePipecatConversation } from "@pipecat-ai/client-react";
import type { ConversationMessage } from "@pipecat-ai/client-react";

function Messages() {
  const { messages } = usePipecatConversation({
    onMessageCreated(message: ConversationMessage) {
      console.log("New message:", message);
    },
    onMessageUpdated(message: ConversationMessage) {
      if (message.final) {
        console.log("Message finalized:", message);
      }
    },
  });

  return (
    <ul>
      {messages.map((msg, i) => (
        <li key={`${msg.createdAt}-${i}`}>
          <strong>{msg.role}:</strong>{" "}
          {msg.parts?.map((part, j) => {
            if (typeof part.text === "string") {
              return <span key={j}>{part.text}</span>;
            }
            // BotOutputText: { spoken, unspoken }
            return (
              <span key={j}>
                <span>{part.text.spoken}</span>
                <span style={{ opacity: 0.5 }}>{part.text.unspoken}</span>
              </span>
            );
          })}
        </li>
      ))}
    </ul>
  );
}

Options

onMessageCreated

(message: ConversationMessage) => void

Called once when a new message first enters the conversation. The message may or may not be complete at this point — check message.final.

onMessageUpdated

(message: ConversationMessage) => void

Called whenever an existing message’s content changes (e.g. streaming text appended, function call status changed, message finalized). Check message.final to detect finalization.

aggregationMetadata

Record<string, AggregationMetadata>

Metadata for aggregation types to control rendering and speech progress behavior. Used to determine which aggregations should be excluded from position-based speech splitting.

Returns

messages

ConversationMessage[]

The current list of conversation messages, ordered for display. Assistant messages have their text parts split into { spoken, unspoken } based on real-time speech progress.

injectMessage

(message: { role: string; parts: ConversationMessagePart[] }) => void

Programmatically inject a message into the conversation (e.g. a system prompt or user-typed input).

useConversationContext

Lower-level hook that provides direct access to the conversation context. Use this when you only need injectMessage without subscribing to the message stream, or to check whether the connected bot supports BotOutput events.

import { useConversationContext } from "@pipecat-ai/client-react";

function TextInput() {
  const { injectMessage, botOutputSupported } = useConversationContext();

  const send = (text: string) => {
    injectMessage({
      role: "user",
      parts: [{ type: "text", text }],
    });
  };

  return (
    <input
      onKeyDown={(e) => e.key === "Enter" && send(e.currentTarget.value)}
    />
  );
}

Returns

injectMessage

(message: { role: string; parts: ConversationMessagePart[] }) => void

Programmatically inject a message into the conversation.

botOutputSupported

boolean | null

Whether the connected bot supports BotOutput events (RTVI 1.1.0+). null means detection hasn’t completed yet.

User interface

These hooks voice-enable a UI: stream the screen to a server-side UIWorker, send UI events, handle the bot’s UI commands, and track background work. They must be used within a PipecatClientProvider. See the RTVI standard for wire payloads and Voice-enabling a client UI for the patterns.

useUISnapshot

Streams accessibility snapshots of the page to the bot. Call once near the root of your app.

useUISnapshot({ enabled: true, debounceMs: 300, trackViewport: true });

Arguments — options?: UseUISnapshotOptions with enabled (default true), debounceMs (default 300), trackViewport (default true), and logSnapshots (default false).

useUIEventSender

Returns (event: string, payload?) => void to send app-defined UI events to the bot (routed to the worker’s @ui_event handlers).

const sendUIEvent = useUIEventSender();
sendUIEvent("note_click", { ref: "e8" });

useUICommandHandler

Registers a handler for a named UI command, active while the component is mounted.

useUICommandHandler("add_note", (payload) => addNoteToPage(payload));

Arguments

command

string

required

handler

(payload: T) => void | Promise<void>

required

Default command handlers

useDefaultUICommandHandlers(options?) installs DOM handlers for all the standard commands at once (scroll_to, focus, highlight, select_text, set_input_value, click). Each resolves the target by snapshot ref, then target_id, and refuses unsafe targets. Install them individually with useDefaultScrollToHandler, useDefaultFocusHandler, useDefaultHighlightHandler, useDefaultSelectTextHandler, useDefaultSetInputValueHandler, and useDefaultClickHandler. For the non-DOM commands, useToastHandler(handler) and useNavigateHandler(handler) are typed convenience wrappers.

useUISnapshot();
useDefaultUICommandHandlers();

useUIJobGroups

Reads accumulated job groups (oldest first) and controls from the nearest UIJobGroupsProvider. Wrap your app in the provider to collect them:

<UIJobGroupsProvider maxGroups={20}>
  <App />
</UIJobGroupsProvider>

const { groups, cancelJobGroup, dismissJobGroup, clearCompleted } =
  useUIJobGroups();

Returns — UIJobGroupsAPI: groups: JobGroup[], cancelJobGroup(jobId, reason?), dismissJobGroup(jobId), and clearCompleted(). Each JobGroup has jobId, label, cancellable, status, and jobs (per-worker { workerName, status, updates, response }).

Documentation Index

​usePipecatClient

​useRTVIClientEvent

​usePipecatClientMediaDevices

​usePipecatClientMediaTrack

​usePipecatClientTransportState

​usePipecatClientCamControl

​usePipecatClientMicControl

​usePipecatConversation

​useConversationContext

​User interface

​useUISnapshot

​useUIEventSender

​useUICommandHandler

​Default command handlers

​useUIJobGroups

usePipecatClient

useRTVIClientEvent

usePipecatClientMediaDevices

usePipecatClientMediaTrack

usePipecatClientTransportState

usePipecatClientCamControl

usePipecatClientMicControl

usePipecatConversation

useConversationContext

User interface

useUISnapshot

useUIEventSender

useUICommandHandler

Default command handlers

useUIJobGroups