XTTS

​

Overview

XTTSService provides text-to-speech capabilities using Coqui’s XTTS (Cross-lingual Text-to-Speech) model through a streaming server. It supports multiple languages and custom voice cloning.

​

Installation

The service requires a running XTTS streaming server. You can start one using Docker:

docker run --gpus=all -e COQUI_TOS_AGREED=1 --rm -p 8000:80 ghcr.io/coqui-ai/xtts-streaming-server:latest-cuda121

For more information, visit the official repository.

​

Configuration

​

Constructor Parameters

​

Output Frames

​

Control Frames

​

Audio Frames

​

Error Frames

​

Methods

See the TTS base class methods for additional functionality.

​

Language Support

Supports multiple languages:

​

Usage Example

from pipecat.services.xtts import XTTSService
from pipecat.transcriptions.language import Language
import aiohttp

# Configure service
async with aiohttp.ClientSession() as session:
    tts_service = XTTSService(
        voice_id="speaker_1",
        language=Language.EN,
        base_url="http://localhost:8000",
        aiohttp_session=session
    )

    # Use in pipeline
    pipeline = Pipeline([
        text_input,         # Produces text
        tts_service,        # Converts text to speech
        audio_output        # Plays audio
    ])

​

Streaming Process

The service handles audio streaming in chunks:

1. Receives audio chunks from XTTS server
2. Buffers chunks for processing
3. Resamples audio to desired sample rate
4. Delivers audio frames in real-time

# Streaming configuration
payload = {
    "text": text,
    "language": language_code,
    "speaker_embedding": embeddings,
    "gpt_cond_latent": latent_data,
    "add_wav_header": False,
    "stream_chunk_size": 20
}

​

Frame Flow

​

Metrics Support

The service collects processing metrics:

• Time to First Byte (TTFB)
• Processing duration
• Character usage
• Streaming performance

​

Notes

• Requires GPU for optimal performance
• Supports real-time streaming
• Automatic audio resampling
• Buffer management for smooth playback
• Thread-safe processing
• Automatic error handling
• Manages server connection lifecycle
• Text preprocessing (removes periods and asterisks)