Converting pipeline frames to RTVI protocol messages
The RTVIObserver
translates Pipecat’s internal pipeline events into standardized RTVI protocol messages. It monitors frame flow through the pipeline and generates corresponding client messages based on event types.
The RTVIObserver
serves two main functions:
The observer is attached to a pipeline task along with the RTVI processor:
The observer maps Pipecat’s internal frames to RTVI protocol messages:
Pipeline Frame | RTVI Message |
---|---|
Speech Events | |
UserStartedSpeakingFrame | RTVIUserStartedSpeakingMessage |
UserStoppedSpeakingFrame | RTVIUserStoppedSpeakingMessage |
BotStartedSpeakingFrame | RTVIBotStartedSpeakingMessage |
BotStoppedSpeakingFrame | RTVIBotStoppedSpeakingMessage |
Transcription | |
TranscriptionFrame | RTVIUserTranscriptionMessage(final=true) |
InterimTranscriptionFrame | RTVIUserTranscriptionMessage(final=false) |
LLM Processing | |
LLMFullResponseStartFrame | RTVIBotLLMStartedMessage |
LLMFullResponseEndFrame | RTVIBotLLMStoppedMessage |
LLMTextFrame | RTVIBotLLMTextMessage |
TTS Events | |
TTSStartedFrame | RTVIBotTTSStartedMessage |
TTSStoppedFrame | RTVIBotTTSStoppedMessage |
TTSTextFrame | RTVIBotTTSTextMessage |
Context/Metrics | |
OpenAILLMContextFrame | RTVIUserLLMTextMessage |
MetricsFrame | RTVIMetricsMessage |
RTVIServerMessageFrame | RTVIServerMessage |
Converting pipeline frames to RTVI protocol messages
The RTVIObserver
translates Pipecat’s internal pipeline events into standardized RTVI protocol messages. It monitors frame flow through the pipeline and generates corresponding client messages based on event types.
The RTVIObserver
serves two main functions:
The observer is attached to a pipeline task along with the RTVI processor:
The observer maps Pipecat’s internal frames to RTVI protocol messages:
Pipeline Frame | RTVI Message |
---|---|
Speech Events | |
UserStartedSpeakingFrame | RTVIUserStartedSpeakingMessage |
UserStoppedSpeakingFrame | RTVIUserStoppedSpeakingMessage |
BotStartedSpeakingFrame | RTVIBotStartedSpeakingMessage |
BotStoppedSpeakingFrame | RTVIBotStoppedSpeakingMessage |
Transcription | |
TranscriptionFrame | RTVIUserTranscriptionMessage(final=true) |
InterimTranscriptionFrame | RTVIUserTranscriptionMessage(final=false) |
LLM Processing | |
LLMFullResponseStartFrame | RTVIBotLLMStartedMessage |
LLMFullResponseEndFrame | RTVIBotLLMStoppedMessage |
LLMTextFrame | RTVIBotLLMTextMessage |
TTS Events | |
TTSStartedFrame | RTVIBotTTSStartedMessage |
TTSStoppedFrame | RTVIBotTTSStoppedMessage |
TTSTextFrame | RTVIBotTTSTextMessage |
Context/Metrics | |
OpenAILLMContextFrame | RTVIUserLLMTextMessage |
MetricsFrame | RTVIMetricsMessage |
RTVIServerMessageFrame | RTVIServerMessage |