​
Overview

SimliVideoService integrates with Simli to create real-time AI avatar video experiences using WebRTC streaming. The service processes audio input to generate synchronized avatar video and audio output, handling real-time streaming, audio resampling, and conversation interruptions for engaging conversational AI applications.

API Reference

Complete API documentation and method details

Simli Docs

Official Simli API documentation and guides

Example Code

Working example with avatar streaming

​
Installation

To use Simli services, install the required dependency: 
pip install "pipecat-ai[simli]"
You’ll also need to set up your Simli credentials as environment variables:
  • SIMLI_API_KEY - Your Simli API key
  • SIMLI_FACE_ID - ID of your avatar face
Sign up for a Simli account at Simli Platform to get your API key and access avatar faces.

​
Frames

​
Input

  • TTSAudioRawFrame - Text-to-speech audio for avatar to speak
  • StartInterruptionFrame - Signals conversation interruption (clears buffer)
  • EndFrame - Signals end of conversation
  • CancelFrame - Signals conversation cancellation

​
Output

  • OutputImageRawFrame - Generated avatar video frames via WebRTC
  • TTSAudioRawFrame - Synchronized audio from the avatar
  • StartInterruptionFrame - Forwarded interruption signals

​
Service Features

  • WebRTC Streaming: Real-time video streaming with low latency
  • Audio Synchronization: Perfect lip-sync with generated speech
  • Buffer Management: Smart audio buffer handling for smooth playback
  • Interruption Handling: Clean conversation interruptions with buffer clearing
  • Automatic Resampling: Handles audio format conversion seamlessly

​
Usage Example

import os
from simli import SimliConfig
from pipecat.services.simli.video import SimliVideoService
from pipecat.pipeline.pipeline import Pipeline

async def main():
    # Configure Simli service
    simli = SimliVideoService(
        SimliConfig(
            apiKey=os.getenv("SIMLI_API_KEY"),
            faceId=os.getenv("SIMLI_FACE_ID"),
            handleSilence=True,      # Keep video active during silence
            maxSessionLength=600,    # 10 minute session limit
            maxIdleTime=30,          # 30 second idle timeout
        ),
        use_turn_server=False,       # Set to True if needed for network
        latency_interval=0,          # Latency monitoring interval
    )

    # Create pipeline with video output
    pipeline = Pipeline([
        transport.input(),              # User input
        stt,                            # Speech-to-text
        context_aggregator.user(),      # User context
        llm,                            # Language model
        tts,                            # Text-to-speech
        simli,                          # Avatar video generation
        transport.output(),             # Video/audio output
        context_aggregator.assistant(), # Assistant context
    ])

    # Configure transport for video (512x512 is optimal for Simli)
    transport_params = DailyParams(
        audio_in_enabled=True,
        audio_out_enabled=True,
        video_out_enabled=True,         # Enable video output
        video_out_is_live=True,         # Real-time video streaming
        video_out_width=512,            # Simli optimized resolution
        video_out_height=512,
        vad_analyzer=SileroVADAnalyzer(),
    )

    # Run the pipeline
    # ...

​
Avatar Configuration

​
Face Selection

Simli supports various avatar faces and emotions: 
from simli import SimliConfig

# Basic avatar configuration
simli_config = SimliConfig(
    apiKey=os.getenv("SIMLI_API_KEY"),
    faceId="your-face-id",           # Your avatar face ID
)

# Avatar with specific emotion (for Trinity faces)
simli_config = SimliConfig(
    apiKey=os.getenv("SIMLI_API_KEY"),
    faceId="faceId/emotionId",       # Face ID with emotion specification
)

​
Session Settings

Configure session behavior and timeouts: 
simli_config = SimliConfig(
    apiKey=os.getenv("SIMLI_API_KEY"),
    faceId=os.getenv("SIMLI_FACE_ID"),
    handleSilence=True,              # Keep sending video during silence
    syncAudio=True,                  # Synchronize audio streams
    maxSessionLength=1200,           # 20 minute maximum session
    maxIdleTime=60,                  # 1 minute idle timeout
)

​
Network Configuration

Configure WebRTC settings for different network environments: 
# For standard networks
simli = SimliVideoService(simli_config, use_turn_server=False)

# For restrictive networks requiring TURN
simli = SimliVideoService(simli_config, use_turn_server=True)

# With latency monitoring (60 second intervals)
simli = SimliVideoService(simli_config, latency_interval=60)

​
Integration Patterns

​
With Daily Transport

Simli works seamlessly with Daily for video conferencing: 
# Daily transport optimized for Simli
transport = DailyTransport(
    room_url=room_url,
    token=token,
    bot_name="AI Avatar",
    params=DailyParams(
        video_out_enabled=True,
        video_out_is_live=True,
        video_out_width=512,         # Simli optimal resolution
        video_out_height=512,
        video_out_framerate=30,
    ),
)

​
With WebRTC Transport

For peer-to-peer video communication: 
# WebRTC transport with Simli settings
transport = SmallWebRTCTransport(
    webrtc_connection=connection,
    params=TransportParams(
        video_out_enabled=True,
        video_out_is_live=True,
        video_out_width=512,
        video_out_height=512,
    ),
)

​
Additional Notes

  • WebRTC Integration: Uses WebRTC for low-latency real-time streaming
  • Network Requirements: Sufficient bandwidth needed for video streaming quality
  • Session Limits: Configure appropriate session lengths based on use case
Simli provides efficient real-time avatar video generation with WebRTC streaming, ideal for applications requiring responsive and engaging visual AI interactions.