C++ SDK Overview

The Pipecat C++ SDK builds native client applications, such as robots, kiosks, smart devices or games, that talk to a Pipecat bot. Like the other client SDKs, it speaks the RTVI protocol, and it:

Starts bots through an endpoint, like Pipecat Cloud or your own server, and connects to them
Sends the user’s audio to the bot and receives the bot’s, as 16-bit PCM
Reports what happens in the conversation through typed callbacks: transcriptions, bot output, function calls, metrics and more
Sends text and app-defined messages to the bot, and waits for its answers

It supports Linux (x86_64 and aarch64), macOS (aarch64) and Windows (x86_64), and needs a C++17 compiler.

Transports

The client connects to your bot with a transport:

Daily

Connects to the bot’s Daily room, using WebRTC

WebSocket

Connects to the bot’s WebSocket

You can also write your own by implementing pipecat::Transport.

Installation

You need CMake 3.16 or newer and a C++17 compiler (GCC 9 or newer, Clang, Apple Clang or MSVC). The client uses libcurl to make HTTP requests, and nlohmann/json 3.7 or newer, which CMake downloads if it’s not installed.

Adding it to your CMake project

The simplest way is to let CMake download a release, with FetchContent. Turn on the transports you need before making it available:

include(FetchContent)

set(PIPECAT_BUILD_WEBSOCKET ON)
FetchContent_Declare(
  pipecat
  URL https://github.com/pipecat-ai/pipecat-client-cxx/releases/download/v1.0.0/pipecat-client-cxx-1.0.0.tar.gz
  URL_HASH SHA256=329deb59b7868be9447d6211c83b276df8f4d3488666c02166dcbf7ec5d27775
)
FetchContent_MakeAvailable(pipecat)

target_link_libraries(my_app PRIVATE pipecat::websocket)

Each transport’s target also links the client. If you only use the client, e.g. with your own transport, link pipecat::pipecat. The client’s dependencies still need to be installed (see below), and so do the transport’s, which its page explains.

Building and installing

You can also build the SDK yourself and install it:

Linux
macOS
Windows

sudo apt-get install cmake ninja-build libcurl4-openssl-dev nlohmann-json3-dev
git clone https://github.com/pipecat-ai/pipecat-client-cxx
cd pipecat-client-cxx
cmake . -G Ninja -Bbuild -DCMAKE_BUILD_TYPE=Release
ninja -C build
cmake --install build --prefix /path/to/pipecat

macOS already includes libcurl:

brew install cmake ninja nlohmann-json
git clone https://github.com/pipecat-ai/pipecat-client-cxx
cd pipecat-client-cxx
cmake . -G Ninja -Bbuild -DCMAKE_BUILD_TYPE=Release
ninja -C build
cmake --install build --prefix /path/to/pipecat

The dependencies come from vcpkg, which downloads them when you configure. Set it up following one of its tutorials, so that VCPKG_ROOT points to it. Then:

git clone https://github.com/pipecat-ai/pipecat-client-cxx
cd pipecat-client-cxx
cmake --preset vcpkg
cmake --build build --config Release
cmake --install build --config Release --prefix C:/path/to/pipecat

Add -DPIPECAT_BUILD_DAILY=ON or -DPIPECAT_BUILD_WEBSOCKET=ON when configuring to build the transports too. To cross-compile for Linux aarch64, see the repository’s README. Then, in your CMake project, point CMAKE_PREFIX_PATH to the install directory, and ask for the transports you use as components:

find_package(pipecat 1.0 REQUIRED COMPONENTS websocket)
target_link_libraries(my_app PRIVATE pipecat::websocket)

A pipecat.pc file is also installed, for Meson and other build systems that use pkg-config.

Usage

Create a pipecat::PipecatClient with a transport and your callbacks, then start the bot and connect to it:

#include <pipecat/daily/transport.h>
#include <pipecat/pipecat.h>

#include <iostream>

class App : public pipecat::PipecatClientCallbacks {
   public:
    void on_bot_ready(const pipecat::rtvi::BotReadyData& data) override {
        std::cout << "Bot ready" << std::endl;
    }

    void on_user_transcript(const pipecat::rtvi::TranscriptData& data) override {
        if (data.final) {
            std::cout << "User: " << data.text << std::endl;
        }
    }

    void on_bot_output(const pipecat::rtvi::BotOutputData& data) override {
        std::cout << "Bot: " << data.text << std::endl;
    }

    // Handle an LLM function call. `respond` can also be called later, from
    // any thread, e.g. when a slow operation finishes.
    void on_llm_function_call_in_progress(
            const pipecat::rtvi::LLMFunctionCallInProgressData& data,
            pipecat::FunctionCallResultCallback respond
    ) override {
        if (data.function_name == "get_weather") {
            respond({{"conditions", "sunny"}, {"temperature", 22}});
        }
    }
};

int main() {
    App app;

    pipecat::PipecatClientOptions options;
    options.transport = std::make_unique<pipecat::DailyTransport>();
    options.callbacks = &app;
    pipecat::PipecatClient client(std::move(options));

    // Start the bot (e.g. on Pipecat Cloud or your own server) and wait
    // until it's ready.
    pipecat::APIRequest request;
    request.endpoint = "https://example.com/start";
    client.start_bot_and_connect(request);

    client.send_text("Hello!");

    // Meanwhile, your audio threads send user audio and play bot audio (16-bit
    // PCM) with client.send_user_audio() and client.read_bot_audio().

    client.disconnect();
}

The client can also send app-defined messages to your bot and wait for its answer, with a callback or a std::future:

auto weather = client.send_client_request("get-weather", {{"city", "SF"}}).get();

Threading

start_bot(), connect() and disconnect() block the calling thread. Everything else returns right away. All methods are thread-safe.
Callbacks and send_client_request() callbacks run on the client’s own event thread, one at a time and in order. You can call any client method from them, including disconnect() or waiting on a send_client_request() future.
Events wait while a callback runs, so keep callbacks short. Function calls can be answered later instead of blocking.
Callbacks must not throw, and must not destroy the client.
Call send_user_audio() and read_bot_audio() from your own audio threads.

Examples

The repository has two examples, which work with a bot on your machine or on Pipecat Cloud, using either transport:

Text

Chat with a bot in the terminal

Voice

Talk with a bot using your microphone and speakers

Migrating from 0.x

Version 1.0 is a rewrite that brings the SDK up to RTVI protocol 2.1, and isn’t compatible with 0.x: rtvi::RTVIClient is now pipecat::PipecatClient, and the Daily transport moved into the same repository. See Migrating from 0.x for where 0.x’s API went.

Documentation

API Reference

The client and transports’ API reference

Transports

Daily and WebSocket transports

Source

Pipecat Client C++ on GitHub

Changelog

What changed in each release

Get Started

Standards

Pipecat Server

Client SDKs

Pipecat Flows

Pipecat Cloud

CLI

Pipecat Context Hub

C++ SDK Overview

Transports

Daily

WebSocket

Installation

Adding it to your CMake project

Building and installing

Usage

Threading

Examples

Text

Voice

Migrating from 0.x

Documentation

API Reference

Transports

Source

Changelog

​Transports

Daily

WebSocket

​Installation

​Adding it to your CMake project

​Building and installing

​Usage

​Threading

​Examples

Text

Voice

​Migrating from 0.x

​Documentation

API Reference

Transports

Source

Changelog

Transports

Installation

Adding it to your CMake project

Building and installing

Usage

Threading

Examples

Migrating from 0.x

Documentation