Skip to main content
Community-maintained integration. This service is built and maintained by Behavioral Signals. Pipecat does not test or officially support it. Please report issues and request changes on the source repository. Learn more about community integrations.

Overview

BehavioralSignalsPerceptionService is a perception producer for expressive conversation. It streams the user’s voice to the Behavioral Signals API and adds per-utterance estimates of expressed emotion, strength, positivity, hesitation, engagement and speaking rate, each with a probability for every class, to the perception of each user turn’s transcriptions. The LLM reads it with the user’s words. Outputs are probabilistic estimates of signals expressed in speech, not determinations of what a person feels, intends or believes. It can run with PipecatToneReader, which reads the words: the service adds its reading to the one a transcription already carries.

Source Repository

Code, an example voice agent, and the changelog

PyPI Package

The pipecat-behavioralsignals package on PyPI

Installation

This is a community-maintained package distributed separately from pipecat-ai:
Requires Python 3.12 or later and Pipecat 1.16.

Prerequisites

  • A Behavioral Signals account, with a client ID (CID) and an API key from the Behavioral Signals portal.
  • Set them as environment variables, or pass them to the service:
The service streams all of the user’s input audio for the whole call, and streaming is billed by the minute. Behavioral Signals saves a stream’s results, which can be fetched later: tell your users, and check that this fits your privacy terms (see Your users’ audio).

Configuration

str | None
default:"None"
Your Behavioral Signals client ID. None uses the BEHAVIORALSIGNALS_CID environment variable.
str | None
default:"None"
Your Behavioral Signals API key. None uses the BEHAVIORALSIGNALS_API_KEY environment variable.

Usage

Put the service right after your STT service, and turn on expressive conversation:
The service reads the audio the STT service passes on, so keep the STT service’s default audio_passthrough=True. It reads mono audio, which Pipecat transports give by default; on audio with more channels it reports an error and stops reading. Each user message then starts with the reading:
A reading covers a recent part of the turn, often not its last words, and a very short turn may have none. The signals are voice_emotion, strength, positivity, speaking_rate, hesitation, engagement and intensity. See the repository for what each reads and how a reading is matched to a turn.

Compatibility

Version 0.2.0 requires Pipecat 1.16 (pipecat-ai>=1.16.0,<1.17) and is tested with 1.16.0. On Pipecat 1.15, use 0.1.0. See the changelog for the versions each release requires.