> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pipecat.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Sophon-Realtime

> Self-hosted, open-weights full-duplex speech-to-speech model, used through the built-in OpenAI Realtime service

> [!NOTE]
>
> **Community-maintained integration.** This service is built and
> maintained by
> [Adventists AI](https://github.com/adventists-ai)
> . Pipecat does not test or officially support it. Please report issues and
> request changes on the
> [source repository](https://github.com/adventists-ai/sophon-realtime)
> . Learn more about
> [community integrations](https://docs.pipecat.ai/api-reference/server/services/community-integrations.md)
> .

## Overview

[Sophon-Realtime](https://github.com/adventists-ai/sophon-realtime) is an
open-weights (Apache-2.0) full-duplex speech-to-speech model family that you
host yourself on one 80 GB GPU with vLLM-Omni. It listens while it talks,
handles barge-in and turn-taking server-side, hears tone (sarcasm, stress,
laughter) and answers with an expressive voice.

Its Live gateway speaks the OpenAI Realtime protocol (GA event names,
function calling, barge-in truncation, PCM16 and G.711), so Pipecat's
built-in `OpenAIRealtimeLLMService` connects to it directly: point `base_url`
at your gateway. No extra package is needed.

Two builds share the same model: **Tag** (the most expressive voice) and
**Flash** (lowest latency, most sessions per GPU).

## Installation

On the Pipecat side, install the OpenAI extra:

```bash theme={null}
uv add "pipecat-ai[openai]"
```

## Prerequisites

A running Sophon-Realtime server and Live gateway (one 80 GB GPU, such as an
H100):

```bash theme={null}
pip install sophon-realtime
sophon-realtime serve --port 8000            # downloads the weights on first run

git clone https://github.com/adventists-ai/sophon-realtime
cd sophon-realtime/live
BUILD=tag PORT=8000 LIVE_PORT=8780 bash start_live.sh
```

The gateway then serves `ws://YOUR_SERVER:8780/v1/realtime`. No API key is
needed unless you set `SOPHON_LIVE_REALTIME_KEYS_FILE` on the server.

## Configuration

Use `OpenAIRealtimeLLMService` with your gateway URL:

```python theme={null}
import os

from pipecat.services.openai.realtime.llm import OpenAIRealtimeLLMService

llm = OpenAIRealtimeLLMService(
    api_key=os.getenv("SOPHON_API_KEY", "any"),
    base_url="ws://YOUR_SERVER:8780/v1/realtime",
    settings=OpenAIRealtimeLLMService.Settings(
        model="sophon-realtime",
        system_instruction="You are a friendly voice assistant.",
    ),
)
```

Turn-taking runs server-side, so the pipeline needs no local VAD or turn
analyzer:

```python theme={null}
pipeline = Pipeline([transport.input(), llm, transport.output()])
```

See the [OpenAI Realtime](https://docs.pipecat.ai/api-reference/server/services/s2s/openai.md) page
for the service's other parameters.

## Compatibility

Tested with Pipecat v1.15.0. Audio is 16-bit mono PCM at 24 kHz in both
directions (G.711 is also accepted).

## Resources

> [card] [Example bot](https://github.com/adventists-ai/sophon-realtime/blob/main/live/examples/pipecat_bot.py)
>
> A runnable Pipecat bot using the development runner

> [card] [Live mode guide](https://github.com/adventists-ai/sophon-realtime/tree/main/live)
>
> Gateway setup, protocol coverage and options

> [card] [Model weights](https://huggingface.co/adventists-ai/Sophon-Realtime-32B-Tag)
>
> Sophon-Realtime-32B-Tag on Hugging Face

> [card] [Try it online](https://www.modelscope.ai/studios/AdventistsAI/Sophon-Realtime)
>
> Talk to it in the browser


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.