Rime

Overview

Rime AI provides two TTS service implementations: RimeTTSService (WebSocket-based) with word-level timing and interruption support, and RimeHttpTTSService (HTTP-based) for simpler use cases. RimeTTSService is recommended for real-time interactive applications.

Rime TTS API Reference

Pipecat’s API methods for Rime TTS integration

Example Implementation

Complete example with word timestamps

Rime Documentation

Official Rime WebSocket and HTTP API documentation

Voice Models

Explore available voice models and features

Installation

To use Rime services, install the required dependencies:

pip install "pipecat-ai[rime]"

Prerequisites

Rime Account Setup

Before using Rime TTS services, you need:

Rime Account: Sign up at Rime AI
API Key: Generate an API key from your account dashboard
Voice Selection: Choose from available voice models

Required Environment Variables

RIME_API_KEY: Your Rime API key for authentication

Configuration

RimeTTSService

api_key

str

required

Rime API key for authentication.

voice_id

str

required

deprecated

ID of the voice to use for synthesis. Deprecated in v0.0.105. Use settings=RimeTTSService.Settings(voice=...) instead.

url

str

default:"wss://users.rime.ai/ws2"

Rime WebSocket API endpoint.

model

str

default:"mistv2"

deprecated

Model ID to use for synthesis. Deprecated in v0.0.105. Use settings=RimeTTSService.Settings(model=...) instead.

sample_rate

int

default:"None"

Output audio sample rate in Hz. When None, uses the pipeline’s configured sample rate.

text_aggregation_mode

TextAggregationMode

default:"TextAggregationMode.SENTENCE"

Controls how incoming text is aggregated before synthesis. SENTENCE (default) buffers text until sentence boundaries, producing more natural speech. TOKEN streams tokens directly for lower latency. Import from pipecat.services.tts_service.

aggregate_sentences

bool

default:"None"

Deprecated in v0.0.104. Use text_aggregation_mode instead.

params

InputParams

default:"None"

deprecated

Deprecated in v0.0.105. Use settings=RimeTTSService.Settings(...) instead.

settings

RimeTTSService.Settings

default:"None"

Runtime-configurable settings. See RimeTTSService Settings below.

RimeHttpTTSService

api_key

str

required

Rime API key for authentication.

voice_id

str

required

deprecated

ID of the voice to use for synthesis. Deprecated in v0.0.105. Use settings=RimeHttpTTSService.Settings(voice=...) instead.

aiohttp_session

aiohttp.ClientSession

required

An aiohttp session for HTTP requests.

model

str

default:"mistv2"

deprecated

Model ID to use for synthesis. Deprecated in v0.0.105. Use settings=RimeHttpTTSService.Settings(model=...) instead.

sample_rate

int

default:"None"

Output audio sample rate in Hz. When None, uses the pipeline’s configured sample rate.

params

InputParams

default:"None"

deprecated

Deprecated in v0.0.105. Use settings=RimeHttpTTSService.Settings(...) instead.

settings

RimeHttpTTSService.Settings

default:"None"

Runtime-configurable settings. See RimeTTSService Settings below.

RimeNonJsonTTSService

A non-JSON WebSocket service for models like Arcana that use plain text messages.

api_key

str

required

Rime API key for authentication.

voice_id

str

required

deprecated

ID of the voice to use for synthesis. Deprecated in v0.0.105. Use settings=RimeNonJsonTTSService.Settings(voice=...) instead.

url

str

default:"wss://users.rime.ai/ws"

Rime WebSocket API endpoint.

model

str

default:"arcana"

deprecated

Model ID to use for synthesis. Deprecated in v0.0.105. Use settings=RimeNonJsonTTSService.Settings(model=...) instead.

audio_format

str

default:"pcm"

Audio output format.

sample_rate

int

default:"None"

Output audio sample rate in Hz. When None, uses the pipeline’s configured sample rate.

text_aggregation_mode

TextAggregationMode

default:"TextAggregationMode.SENTENCE"

Controls how incoming text is aggregated before synthesis. SENTENCE (default) buffers text until sentence boundaries. TOKEN streams tokens directly for lower latency. Import from pipecat.services.tts_service.

aggregate_sentences

bool

default:"None"

deprecated

Deprecated in v0.0.104. Use text_aggregation_mode instead.

params

InputParams

default:"None"

deprecated

Deprecated in v0.0.105. Use settings=RimeNonJsonTTSService.Settings(...) instead.

settings

RimeNonJsonTTSService.Settings

default:"None"

Runtime-configurable settings. See RimeNonJsonTTSService Settings below.

RimeTTSService Settings

Runtime-configurable settings passed via the settings constructor argument using RimeTTSService.Settings(...). These can be updated mid-conversation with TTSUpdateSettingsFrame. See Service Settings for details.

Parameter	Type	Default	Description
`model`	`str`	`None`	Model identifier. (Inherited.)
`voice`	`str`	`None`	Voice identifier. (Inherited.)
`language`	`Language \| str`	`None`	Language for synthesis. (Inherited.)
`segment`	`str`	`NOT_GIVEN`	Segment type for synthesis.
`speedAlpha`	`float`	`NOT_GIVEN`	Speed alpha parameter.
`reduceLatency`	`bool`	`NOT_GIVEN`	Whether to reduce latency.
`pauseBetweenBrackets`	`bool`	`NOT_GIVEN`	Pause between brackets.
`phonemizeBetweenBrackets`	`bool`	`NOT_GIVEN`	Phonemize between brackets.
`noTextNormalization`	`bool`	`NOT_GIVEN`	Disable text normalization.
`saveOovs`	`bool`	`NOT_GIVEN`	Save out-of-vocabulary words.
`inlineSpeedAlpha`	`str`	`NOT_GIVEN`	Inline speed alpha.
`repetition_penalty`	`float`	`NOT_GIVEN`	Repetition penalty.
`temperature`	`float`	`NOT_GIVEN`	Temperature for sampling.
`top_p`	`float`	`NOT_GIVEN`	Top-p sampling parameter.

RimeNonJsonTTSService Settings

Runtime-configurable settings passed via the settings constructor argument using RimeNonJsonTTSService.Settings(...). These can be updated mid-conversation with TTSUpdateSettingsFrame. See Service Settings for details.

Parameter	Type	Default	Description
`model`	`str`	`None`	Model identifier. (Inherited.)
`voice`	`str`	`None`	Voice identifier. (Inherited.)
`language`	`Language \| str`	`None`	Language for synthesis. (Inherited.)
`segment`	`str`	`NOT_GIVEN`	Segment type for synthesis.
`repetition_penalty`	`float`	`NOT_GIVEN`	Repetition penalty.
`temperature`	`float`	`NOT_GIVEN`	Temperature for sampling.
`top_p`	`float`	`NOT_GIVEN`	Top-p sampling parameter.

Usage

Basic Setup (WebSocket)

from pipecat.services.rime import RimeTTSService

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    settings=RimeTTSService.Settings(
        voice="cove",
    ),
)

With Customization (WebSocket)

from pipecat.services.rime import RimeTTSService
from pipecat.transcriptions.language import Language

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    settings=RimeTTSService.Settings(
        voice="cove",
        model="mistv2",
        language=Language.ES,
        speedAlpha=1.2,
        reduceLatency=True,
    ),
)

HTTP Service

import aiohttp
from pipecat.services.rime import RimeHttpTTSService

async with aiohttp.ClientSession() as session:
    tts = RimeHttpTTSService(
        api_key=os.getenv("RIME_API_KEY"),
        settings=RimeHttpTTSService.Settings(
            voice="cove",
        ),
        aiohttp_session=session,
    )

Non-JSON WebSocket (Arcana)

from pipecat.services.rime import RimeNonJsonTTSService

tts = RimeNonJsonTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    settings=RimeNonJsonTTSService.Settings(
        voice="cove",
        model="arcana",
    ),
)

Customizing Speech

RimeTTSService provides a set of helper methods for implementing Rime-specific customizations, meant to be used as part of text transformers. These include methods for spelling out text, adjusting speech rate, and modifying pitch. See the Text Transformers for TTS section in the Text-to-Speech guide for usage examples.

SPELL(text: str) -> str:

Implements Rime’s spell function to spell out text character by character.

# Text transformers for TTS
# This will insert Rime's spell tags around the provided text.
async def spell_out_text(text: str, type: str) -> str:
    return RimeTTSService.SPELL(text)

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    text_transforms=[
        ("phone_number", spell_out_text),
    ],
)

PAUSE_TAG(seconds: float) -> str:

Implements Rime’s custom pause functionality to generate a properly formatted pause tag you can insert into the text.

# Text transformers for TTS
# This will insert a one second pause after questions.
async def pause_after_questions(text: str, type: str) -> str:
    if text.endswith("?"):
        return f"{text}{RimeTTSService.PAUSE_TAG(1.0)}"
    return text

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    text_transforms=[
        ("sentence", pause_after_questions), # Only apply to sentence aggregations
    ],
)

PRONOUNCE(self, text: str, word: str, phoneme: str) -> str:

Convenience method to support Rime’s custom pronunciations feature. It takes a word and its desired phoneme representation, returning the text with the provided word replaced by the appropriate phoneme tag.

# Text transformers for TTS
# This will a phoneme in place of the word "potato" to define how it
# should be pronounced.
async def maybe_say_potato_all_fancylike(text: str, type: str) -> str:
    if using_fancy_voice:
        return RimeTTSService.PRONOUNCE(text, "potato", "potato")
    else:
        return RimeTTSService.PRONOUNCE(text, "potato", "poteto")

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    text_transforms=[
        ("*", maybe_say_potato_all_fancylike), # Apply to all text
    ],
)

INLINE_SPEED(self, text: str, speed: float) -> str:

A convenience method to support Rime’s inline speed adjustment feature. It will wrap the provided text in the [] tags and add the provided speed to the inlineSpeedAlpha field in the request metadata.

# Text transformers for TTS
# This will make the word slow always be spoken more slowly.
async def slow_down_slow_words(text: str, type: str) -> str:
    return text.replace(
        "slow",
        RimeTTSService.INLINE_SPEED("slow", speed=0.5)
    )

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    text_transforms=[
        ("*", slow_down_slow_words), # Apply to all text
    ],
)

The InputParams / params= pattern is deprecated as of v0.0.105. Use Settings / settings= instead. See the Service Settings guide for migration details.

Notes

Word-level timestamps: RimeTTSService provides word-level timing information, enabling synchronized text highlighting.
WebSocket vs HTTP: The WebSocket service supports word-level timestamps, interruption handling, and maintains context across messages within a turn. The HTTP service is simpler but lacks these features.
Non-JSON WebSocket: RimeNonJsonTTSService is for models like Arcana that use plain text messages instead of JSON. It does not support word-level timestamps.

Event Handlers

Rime WebSocket TTS services support the standard service connection events:

Event	Description
`on_connected`	Connected to Rime WebSocket
`on_disconnected`	Disconnected from Rime WebSocket
`on_connection_error`	WebSocket connection error occurred

@tts.event_handler("on_connected")
async def on_connected(service):
    print("Connected to Rime")

API Reference

Services

Utilities

Frameworks

Pipeline

Overview

Rime TTS API Reference

Example Implementation

Rime Documentation

Voice Models

Installation

Prerequisites

Rime Account Setup

Required Environment Variables

Configuration

RimeTTSService

RimeHttpTTSService

RimeNonJsonTTSService

RimeTTSService Settings

RimeNonJsonTTSService Settings

Usage

Basic Setup (WebSocket)

With Customization (WebSocket)

HTTP Service

Non-JSON WebSocket (Arcana)

Customizing Speech

SPELL(text: str) -> str:

PAUSE_TAG(seconds: float) -> str:

PRONOUNCE(self, text: str, word: str, phoneme: str) -> str:

INLINE_SPEED(self, text: str, speed: float) -> str:

Notes

Event Handlers

API Reference

Services

Utilities

Frameworks

Pipeline

​Overview

Rime TTS API Reference

Example Implementation

Rime Documentation

Voice Models

​Installation

​Prerequisites

​Rime Account Setup

​Required Environment Variables

​Configuration

​RimeTTSService

​RimeHttpTTSService

​RimeNonJsonTTSService

​RimeTTSService Settings

​RimeNonJsonTTSService Settings

​Usage

​Basic Setup (WebSocket)

​With Customization (WebSocket)

​HTTP Service

​Non-JSON WebSocket (Arcana)

​Customizing Speech

​SPELL(text: str) -> str:

​PAUSE_TAG(seconds: float) -> str:

​PRONOUNCE(self, text: str, word: str, phoneme: str) -> str:

​INLINE_SPEED(self, text: str, speed: float) -> str:

​Notes

​Event Handlers

Overview

Installation

Prerequisites

Rime Account Setup

Required Environment Variables

Configuration

RimeTTSService

RimeHttpTTSService

RimeNonJsonTTSService

RimeTTSService Settings

RimeNonJsonTTSService Settings

Usage

Basic Setup (WebSocket)

With Customization (WebSocket)

HTTP Service

Non-JSON WebSocket (Arcana)

Customizing Speech

SPELL(text: str) -> str:

PAUSE_TAG(seconds: float) -> str:

PRONOUNCE(self, text: str, word: str, phoneme: str) -> str:

INLINE_SPEED(self, text: str, speed: float) -> str:

Notes

Event Handlers