Skip to main content
Community-maintained integration. This service is built and maintained by marujofln. Pipecat does not test or officially support it. Please report issues and request changes on the source repository. Learn more about community integrations.

Overview

FreeSWITCHFrameSerializer connects a Pipecat pipeline to a FreeSWITCH call through the open source mod_audio_stream module (community edition). Caller audio arrives over the module’s WebSocket and becomes InputAudioRawFrame; bot audio is sent back as streamAudio blocks. The community edition of mod_audio_stream only writes each bot audio block to a file, it does not play it. The serializer therefore keeps one FreeSWITCH event socket (ESL) connection per call: it queues each file for playback, stops playback on interruption (uuid_break), hangs up the call (uuid_kill) after the queued audio has played, and delivers the keys the caller presses (DTMF) as InputDTMFFrame. It is designed to be used with Pipecat’s FastAPIWebsocketTransport. With the serializer’s optional lookahead processor in the pipeline, bot speech is sent to FreeSWITCH ahead of the transport’s real-time clock, in blocks that grow while the bot speaks, so a few playbacks carry a whole utterance.

Source Repository

Source code, the example bot, a Docker test bench and issues

PyPI Package

The pipecat-freeswitch package on PyPI

FreeSWITCH

The open source telephony platform

mod_audio_stream

The FreeSWITCH module that streams call audio over a WebSocket

Installation

This is a community-maintained package distributed separately from pipecat-ai:
Or use pip install pipecat-freeswitch if you manage dependencies with pip.

Prerequisites

FreeSWITCH is self-hosted, so there is no provider account to configure. You need:
  1. FreeSWITCH with mod_audio_stream (community edition) loaded. The dialplan answers the call and starts the stream with the channel UUID as metadata: uuid_audio_stream ${uuid} start ws://BOT_HOST:8765/ws mono 8k ${uuid}.
  2. The event socket (mod_event_socket) reachable from the bot, restricted with an ACL and a dedicated password: it grants full control of FreeSWITCH.
  3. A token on the WebSocket handshake (recommended): FreeSWITCH can send Authorization: Bearer <token> through the STREAM_EXTRA_HEADERS channel variable, and your endpoint checks it before accepting the connection. Use wss:// or a private network.
The repository includes a Docker test bench (FreeSWITCH 1.11.3 with mod_audio_stream) and a complete example bot. See the source repository for the dialplan, the security setup and the example.

Configuration

str
default:"None"
UUID of the FreeSWITCH channel. When omitted, it is taken from the first text message mod_audio_stream sends. Only canonical UUIDs are accepted.
str
default:"127.0.0.1"
FreeSWITCH event socket host.
int
default:"8021"
FreeSWITCH event socket port.
str
default:"ClueCon"
Event socket password. A warning is logged when FreeSWITCH’s default is used.
FreeSWITCHFrameSerializer.InputParams
default:"None"
Optional settings, including freeswitch_sample_rate (the rate for both directions, 8000 by default), freeswitch_in_sample_rate and freeswitch_out_sample_rate (per-direction overrides: the rate given to uuid_audio_stream, and the channel’s codec rate: 8000, 16000, 24000, 32000 or 48000), output_gain_db, auto_hang_up, hang_up_timeout_secs and lookahead_margin_secs (with the lookahead processor, how far ahead of the transport’s clock the audio already sent must be before the next block is held back; 0.1 by default).
This serializer does not use the Pipecat Settings dataclass pattern. See the source repository for the authoritative, up-to-date parameters.

Usage

Read the channel UUID from the first message, pass the serializer to a FastAPIWebsocketTransport, and call serializer.cleanup() when the WebSocket closes:
Every playback that FreeSWITCH starts flushes about 30-40 ms of the caller’s incoming audio, so many short blocks keep the bot from hearing barge-ins. Place serializer.lookahead() right before transport.output(), after everything that produces bot audio: the speech then reaches FreeSWITCH ahead of time in blocks that grow, and audio_out_10ms_chunks only sets the first one. Without it, use output blocks of 200 ms or more (audio_out_10ms_chunks=20). Send bot audio at the channel’s codec rate to avoid resampling artifacts. See the repository’s README for the measurements.
With the lookahead processor, only audio that passes through it is played: the transport’s output mixer, its end silence and the tones of OutputDTMFFrame are not.

Compatibility

Tested with Pipecat v1.14.0, FreeSWITCH 1.11.3, mod_audio_stream commit da8b068 (community edition) and Python 3.12 and 3.13, with PCMU and Opus calls. The unit and integration tests also pass on Pipecat v1.16.0, which requires version 0.3.0 or later. Growing blocks (serializer.lookahead()) require version 0.3.0 or later. DTMF requires version 0.2.0 or later: the keys 0-9, * and # become InputDTMFFrame when the phone sends them as RFC 2833 telephone events or SIP INFO. In-band tones are only detected if the dialplan runs start_dtmf before uuid_audio_stream. Check the source repository for the latest tested versions and the changelog.