​
Voice Assistants

Pipecat makes it easy to build voice-based AI agents that can:

  • Listen to user speech and convert it to text
  • Maintain conversation context across multiple exchanges
  • Generate appropriate responses using LLMs
  • Convert responses back to natural-sounding speech
  • Handle all of this in real-time for natural conversations

Rather than dealing with the complexity of coordinating multiple AI services and managing real-time audio, Pipecat handles the orchestration for you. You can focus on defining your agent’s behavior and let Pipecat manage the technical details of real-time processing and service integration.

# Example voice assistant pipeline
# (We'll explain how this works in detail in later sections)
pipeline = Pipeline([
    transport.input(),          # Audio input
    transcription_service,      # Speech-to-text
    llm_context_aggregator,     # Manages conversation context
    llm_service,                # Processes with LLM
    tts_service,                # Text-to-speech
    transport.output()          # Audio output
])

​
Multimodal Applications

Pipecat excels at handling multiple data types simultaneously:

  • Audio streams for voice interaction
  • Video frames for visual processing
  • Text for LLM interaction
  • Generated images for visual responses

​
Real-time AI Processing

Built to handle streaming AI workloads:

  • Continuous speech recognition
  • Real-time LLM interactions
  • Dynamic audio/video generation
  • Interactive media processing
OverviewCore Concepts