Skip to main content
The RTVIObserver translates Pipecat’s internal pipeline events into standardized RTVI protocol messages. It monitors frame flow through the pipeline and generates corresponding client messages based on event types.

​
Purpose

The RTVIObserver primarily serves to convert internal pipeline frames into to client-compatible RTVI messages. It is required for any application using RTVI as the client protocol to ensure proper communication of events such as speech start/stop, user transcript, bot output, metrics, and server messages.

​
Adding to a Pipeline

The observer is attached to a pipeline task along with the RTVI processor: 
# Create the RTVIProcessor
rtvi = RTVIProcessor()

# Add to pipeline
pipeline = Pipeline([
    transport.input(),
    rtvi,
    # Other processors...
])

# Create pipeline task with observer
task = PipelineTask(
    pipeline,
    params=PipelineParams(allow_interruptions=True),
    observers=[RTVIObserver(rtvi)],  # Add the observer here
)

​
Configuration

The RTVIObserver should be initialized with the RTVIProcessor instance along with an optional set of parameters with the following fields:
​
bot_output_enabled
default:"True"
Indicates if bot output messages should be sent.
​
bot_llm_enabled
default:"True"
Indicates if the bot’s LLM messages should be sent.
​
bot_tts_enabled
default:"True"
Indicates if the bot’s TTS messages should be sent.
​
bot_speaking_enabled
default:"True"
Indicates if the bot’s started/stopped speaking messages should be sent.
​
bot_audio_level_enabled
default:"False"
Indicates if bot’s audio level messages should be sent.
​
user_llm_enabled
default:"True"
Indicates if the user’s LLM input messages should be sent.
​
user_speaking_enabled
default:"True"
Indicates if the user’s started/stopped speaking messages should be sent.
​
user_transcription_enabled
default:"True"
Indicates if user’s transcription messages should be sent.
​
user_audio_level_enabled
default:"False"
Indicates if user’s audio level messages should be sent.
​
metrics_enabled
default:"True"
Indicates if metrics messages should be sent.
​
system_logs_enabled
default:"False"
Indicates if system logs should be sent.
​
errors_enabled
default:"True"
⚠️ Deprecated: Indicates if errors messages should be sent.
​
skip_aggregator_types
default:"None"
List of aggregation types to skip sending as tts/output messages.
If using this to avoid sending secure information, be sure to also disable bot_llm_enabled to avoid leaking through LLM messages.
​
bot_output_transforms
default:"None"
A list of tuples to transform text just before sending it to TTS. Each tuple should be of the form (aggregation_type, transform_function), where aggregation_type is a string (or '*' for all types), and transform_function is a callable that takes (text, aggregation_type) and returns the transformed text.Example:
def redact_sensitive(text, agg_type):
    # Example: redact numbers
    import re
    return re.sub(r"\d+", "[REDACTED]", text)

bot_output_transforms = [
    ("credit_card", redact_sensitive),  # Only for 'credit_card' type
    ("*", lambda text, agg_type: text.upper()),  # For all types, make uppercase
]

observer = RTVIObserver(
    rtvi,
    params=RTVIObserverParams(bot_output_transforms=bot_output_transforms),
)
​
audio_level_period_secs
default:"0.15"
How often audio levels should be sent if enabled.

​
Frame Translation

The observer maps Pipecat’s internal frames to RTVI protocol messages:
Pipeline FrameRTVI Message
Speech Events
UserStartedSpeakingFrameRTVIUserStartedSpeakingMessage
UserStoppedSpeakingFrameRTVIUserStoppedSpeakingMessage
BotStartedSpeakingFrameRTVIBotStartedSpeakingMessage
BotStoppedSpeakingFrameRTVIBotStoppedSpeakingMessage
Transcription
TranscriptionFrameRTVIUserTranscriptionMessage(final=true)
InterimTranscriptionFrameRTVIUserTranscriptionMessage(final=false)
Bot Output
AggregatedTextFrameRTVIBotOutputMessage
LLM Processing
LLMFullResponseStartFrameRTVIBotLLMStartedMessage
LLMFullResponseEndFrameRTVIBotLLMStoppedMessage
LLMTextFrameRTVIBotLLMTextMessage
TTS Events
TTSStartedFrameRTVIBotTTSStartedMessage
TTSStoppedFrameRTVIBotTTSStoppedMessage
TTSTextFrameRTVIBotTTSTextMessage
Context/Metrics
LLMContextFrameRTVIUserLLMTextMessage
MetricsFrameRTVIMetricsMessage
RTVIServerMessageFrameRTVIServerMessage