Overview

Pipecat integrates with Tavus to support AI-generated video avatars. The integration includes:
  • TavusVideoService – a pipeline service that handles audio streaming and requests Tavus to generate avatar video responses.
  • TavusTransport – a transport layer that directly streams generated Tavus video to users in real-time. All the details here
For full integration details, visit Tavus + Pipecat Docs.

Installation

Install the Tavus integration for Pipecat:
pip install "pipecat-ai[tavus]"
Set the following environment variables:
  • TAVUS_API_KEY – Your Tavus API key
  • TAVUS_REPLICA_ID – Tavus replica identifier
  • (Optional) TAVUS_PERSONA_ID – Persona ID for video generation
Sign up to get your API key at Tavus.

TavusVideoService

Purpose

Handles real-time audio and coordinates Tavus avatar video generation.

Constructor Parameters

api_key
str
required
Tavus API key
replica_id
str
required
Tavus replica identifier
persona_id
str
default:"pipecat-stream"
Optional persona ID (default is pipecat-stream)
session
aiohttp.ClientSession
required
HTTP client session for API communication

Input Frames

Audio Input

TTSAudioRawFrame
Frame
Raw audio data for avatar speech

Control Frames

StartInterruptionFrame
Frame
Signals conversation interruption
EndFrame
Frame
Signals end of conversation
CancelFrame
Frame
Signals conversation cancellation

Usage Example

from pipecat.services.tavus.video import TavusVideoService
import aiohttp

async with aiohttp.ClientSession() as session:
    # Configure service
    tavus = TavusVideoService(
        api_key=os.getenv("TAVUS_API_KEY"),
        replica_id=os.getenv("TAVUS_REPLICA_ID"),
        session=session,
    )

    # Use in pipeline
    pipeline = Pipeline(
        [
            transport.input(),                # Transport user input
            stt,                              # STT
            context_aggregator.user(),        # User responses
            llm,                              # LLM
            tts,                              # TTS
            tavus,                            # Tavus output layer
            transport.output(),               # Transport bot output
            context_aggregator.assistant(),   # Assistant spoken responses
        ]
    )

Frame Flow

Metrics Support

The service collects processing metrics:
  • Processing duration
  • Time to First Byte (TTFB)
  • API response times
  • Audio processing metrics

Common Use Cases

  1. AI Video Avatars
    • Virtual assistants
    • Customer service
    • Educational content
  2. Interactive Presentations
    • Product demonstrations
    • Training materials
    • Marketing content
  3. Real-time Communication
    • Video conferencing
    • Virtual meetings
    • Interactive broadcasts

Notes

  • Handles real-time audio streaming
  • Supports conversation interruptions
  • Manages conversation lifecycle
  • Automatic audio resampling
  • Thread-safe processing
  • Includes comprehensive error handling