The UserIdleProcessor is a specialized frame processor that monitors user activity in a conversation and executes callbacks when the user becomes idle. It’s particularly useful for maintaining engagement by detecting periods of user inactivity and providing escalating responses to inactivity.

​
Constructor Parameters

​
callback
Union[BasicCallback, RetryCallback]
required

An async function that will be called when user inactivity is detected. Can be either:

  • Basic callback: async def(processor: UserIdleProcessor) -> None

  • Retry callback: async def(processor: UserIdleProcessor, retry_count: int) -> bool where returning False stops idle monitoring

​
timeout
float
required

The number of seconds to wait before considering the user idle.

​
Behavior

The processor starts monitoring for inactivity only after the first conversation activity (either UserStartedSpeakingFrame or BotSpeakingFrame). It manages idle state based on the following rules:

  • Resets idle timer when user starts or stops speaking
  • Pauses idle monitoring while user is speaking
  • Resets idle timer when bot is speaking
  • Stops monitoring on conversation end or cancellation
  • Manages a retry count for the retry callback
  • Stops monitoring when retry callback returns False

​
Properties

​
retry_count
int

The current number of retry attempts made to engage the user.

​
Example Implementations

Here are two example showing how to use the UserIdleProcessor: one with the basic callback and one with the retry callback:

async def handle_idle(user_idle: UserIdleProcessor) -> None:
    messages.append({
        "role": "system",
        "content": "Ask the user if they are still there and try to prompt for some input."
    })
    await user_idle.push_frame(LLMMessagesFrame(messages))

# Create the processor
user_idle = UserIdleProcessor(
    callback=handle_idle,
    timeout=5.0
)

# Add to pipeline
pipeline = Pipeline([
    transport.input(),
    user_idle,  # Add the processor to monitor user activity
    context_aggregator.user(),
    # ... rest of pipeline
])

​
Frame Handling

The processor handles the following frame types:

  • UserStartedSpeakingFrame: Marks user as active, resets idle timer and retry count
  • UserStoppedSpeakingFrame: Starts idle monitoring
  • BotSpeakingFrame: Resets idle timer
  • EndFrame / CancelFrame: Stops idle monitoring

​
Notes

  • The idle callback won’t be triggered while the user or bot is actively speaking
  • The processor automatically cleans up its resources when the pipeline ends
  • Basic callbacks are supported for backward compatibility
TranscriptProcessor