> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pipecat.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Eval Conversation Observer

> EvalConversationObserver records a bot's conversation in the file format eval simulations save, so a real call can be judged with pipecat eval judge.

The `EvalConversationObserver` records a bot's conversation in the same `<stem>.conversation.json` format that `pipecat eval` simulation runs save. This lets you judge a call the eval harness didn't drive, such as a production call, against a simulation's `success` criterion and metrics with `pipecat eval judge`.

## What Gets Recorded

Each side's turn is what the context aggregators write to the LLM context:

* **User turns**: What the bot heard. Text a client sends with RTVI's `send-text` is also recorded as a user turn.
* **Bot turns**: What the bot said, cut where the user interrupted it.
* **Function calls**: Each call the bot makes, recorded in place in the conversation. Calls that are cancelled are removed.

Each turn is recorded as it is written, so summarizing the context later doesn't lose it. The context's other messages, such as tool changes made by Pipecat Flows, are left out.

The observer records what passes through the worker it is attached to. In a bot with several workers, attach it to the worker holding the conversation's context; calls made in other workers are not recorded.

## Events

The observer emits one event:

* **`on_conversation_ended`**: Emitted once the pipeline is done
  * Parameters: `observer` (EvalConversationObserver), `message` (EvalConversationEndedMessage)

## Usage

```python theme={null}
import json
from pathlib import Path

from pipecat.evals.conversation_observer import EvalConversationObserver
from pipecat.pipeline.worker import PipelineWorker

observer = EvalConversationObserver()

@observer.event_handler("on_conversation_ended")
async def on_conversation_ended(observer, message):
    Path("call.conversation.json").write_text(json.dumps(message.conversation, indent=2))

worker = PipelineWorker(
    pipeline,
    observers=[observer],
)
```

Then judge the saved file against a simulation scenario:

```bash theme={null}
pipecat eval judge scenarios/<name>.yaml -s <scenario> call.conversation.json
```

## EvalConversationEndedMessage

<ParamField path="conversation" type="list[dict]" required>
  The conversation's messages, in the format of a conversation file.
</ParamField>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.