AWS Nova Sonic

Overview

AWSNovaSonicLLMService enables natural, real-time conversations with AWS Nova Sonic. It provides built-in audio transcription, voice activity detection, and context management for creating interactive AI experiences with bidirectional audio streaming, text generation, and function calling capabilities.

AWS Nova Sonic API Reference

Pipecat’s API methods for AWS Nova Sonic integration

Example Implementation

Complete AWS Nova Sonic conversation example

AWS Bedrock Documentation

Official AWS Bedrock and Nova Sonic documentation

AWS Console

Access AWS Bedrock and manage Nova Sonic models

Installation

To use AWS Nova Sonic services, install the required dependencies:

uv add "pipecat-ai[aws-nova-sonic]"

Prerequisites

AWS Account Setup

Before using AWS Nova Sonic services, you need:

AWS Account: Set up at AWS Console
Bedrock Access: Enable AWS Bedrock service in your region
Model Access: Request access to Nova Sonic models in Bedrock
IAM Credentials: Configure AWS access keys with Bedrock permissions

Required Environment Variables

AWS_SECRET_ACCESS_KEY: Your AWS secret access key
AWS_ACCESS_KEY_ID: Your AWS access key ID
AWS_REGION: AWS region where Bedrock is available

Key Features

Real-time Speech-to-Speech: Direct audio input to audio output processing
Built-in Transcription: Automatic speech-to-text with real-time streaming
Voice Activity Detection: Automatic detection of speech start/stop
Function Calling: Support for external function and API integration
Multiple Voices: See AWS Nova 2 Sonic voice support for available voices

Configuration

AWSNovaSonicLLMService

str

required

AWS secret access key for authentication.

str

required

AWS access key ID for authentication.

str

default:"None"

AWS session token for temporary credentials (e.g., when using AWS STS).

str

required

AWS region where the service is hosted. Supported regions: "us-east-1", "us-west-2", "eu-north-1", "ap-northeast-1".

str

default:"amazon.nova-2-5-sonic"

deprecated

Deprecated since 0.0.105, removed in 2.0.0. Use settings=AWSNovaSonicLLMService.Settings(model=...) instead.

str

default:"matthew"

deprecated

Deprecated since 0.0.105, removed in 2.0.0. Use settings=AWSNovaSonicLLMService.Settings(voice=...) instead.

Params

default:"Params()"

deprecated

Deprecated since 0.0.105, removed in 2.0.0. Use settings=AWSNovaSonicLLMService.Settings(...) for inference settings and audio_config=AudioConfig(...) for audio configuration.

AudioConfig

default:"None"

Audio configuration (sample rates, sample sizes, channel counts). If not provided, defaults are used (16kHz input, 24kHz output, 16-bit, mono). See AudioConfig below.

AWSNovaSonicLLMService.Settings

default:"None"

Inference and voice settings. See Settings below.

str

default:"None"

deprecated

Deprecated since 0.0.105, removed in 2.0.0. Use settings=AWSNovaSonicLLMService.Settings(system_instruction=...) instead.

ToolsSchema | List[FunctionSchema | DirectFunction]

default:"None"

Available tools for the model: a ToolsSchema, or a plain list of direct functions and/or FunctionSchema objects.

SessionContinuationParams

default:"None"

Configuration for automatic session continuation. When enabled (the default), sessions are seamlessly rotated before the AWS time limit (~8 minutes) with no user-perceptible interruption. See SessionContinuationParams below.

Settings

Settings passed via the settings constructor argument using AWSNovaSonicLLMService.Settings(...). Changing them mid-conversation with LLMUpdateSettingsFrame is not supported: the service stores the change and logs a warning, but doesn’t apply it to the session in progress. See Service Settings for details.

Parameter	Type	Default	Description
`model`	`str`	`"amazon.nova-2-5-sonic"`	Model identifier. (Inherited from base settings.)
`system_instruction`	`str`	`None`	System instruction/prompt. (Inherited from base settings.)
`temperature`	`float`	`0.7`	Sampling temperature for text generation. (Inherited from base settings.)
`max_tokens`	`int`	`1024`	Maximum number of tokens to generate. (Inherited from base settings.)
`top_p`	`float`	`0.9`	Nucleus sampling parameter. (Inherited from base settings.)
`voice`	`str`	`"matthew"`	Voice ID for speech synthesis.
`endpointing_sensitivity`	`str \| None`	`None`	Controls how quickly Nova Sonic decides the user has stopped speaking. Values: `"LOW"`, `"MEDIUM"`, or `"HIGH"` (responds most quickly). Supported with Nova 2 Sonic and later models (including the default Nova 2.5 Sonic); ignored with a warning on `amazon.nova-sonic-v1:0`.

When endpointing_sensitivity is None it is omitted, so the model’s default applies. temperature, max_tokens, and top_p are always sent and must not be None.

AudioConfig

Audio configuration passed via the audio_config constructor argument.

Parameter	Type	Default	Description
`input_sample_rate`	`int`	`16000`	Audio input sample rate in Hz.
`input_sample_size`	`int`	`16`	Audio input sample size in bits.
`input_channel_count`	`int`	`1`	Number of input audio channels.
`output_sample_rate`	`int`	`24000`	Audio output sample rate in Hz.
`output_sample_size`	`int`	`16`	Audio output sample size in bits.
`output_channel_count`	`int`	`1`	Number of output audio channels.

SessionContinuationParams

Configuration for automatic session continuation, passed via the session_continuation constructor argument. Nova Sonic sessions have an AWS-imposed time limit (~8 minutes). When enabled, session continuation proactively creates a new session in the background before the limit is reached, buffers user audio during the transition, and hands off to it. The new session gets the conversation as transcript text (the initial context plus what the user and assistant have said since, up to the most recent 200 KB), so function call results and messages added with LLMMessagesAppendFrame don’t carry over.

Parameter	Type	Default	Description
`enabled`	`bool`	`True`	Whether automatic session continuation is enabled.
`transition_threshold_seconds`	`float`	`360.0`	How many seconds into a session to begin monitoring for a transition opportunity. The service switches to a new session when the assistant’s next response after this threshold ends.
`audio_buffer_duration_seconds`	`float`	`3.0`	Duration of the rolling audio buffer (in seconds) that captures user audio during the transition window. This audio is replayed into the new session so no user input is lost.
`audio_start_timeout_seconds`	`float`	`80.0`	How long to wait for the assistant to start speaking after the threshold is reached. When it expires, the service doesn’t switch sessions: it stays on the current one and keeps waiting for the assistant to speak. Set to `0` to disable the timeout.

Usage

Basic Setup

import os
from pipecat.services.aws.nova_sonic.llm import AWSNovaSonicLLMService

llm = AWSNovaSonicLLMService(
    secret_access_key=os.getenv("AWS_SECRET_ACCESS_KEY"),
    access_key_id=os.getenv("AWS_ACCESS_KEY_ID"),
    region=os.getenv("AWS_REGION"),
    settings=AWSNovaSonicLLMService.Settings(
        voice="matthew",
        system_instruction="You are a helpful assistant.",
    ),
)

With Settings

from pipecat.services.aws.nova_sonic.llm import AWSNovaSonicLLMService, AudioConfig

llm = AWSNovaSonicLLMService(
    secret_access_key=os.getenv("AWS_SECRET_ACCESS_KEY"),
    access_key_id=os.getenv("AWS_ACCESS_KEY_ID"),
    region="us-east-1",
    audio_config=AudioConfig(
        input_sample_rate=16000,
        output_sample_rate=24000,
    ),
    settings=AWSNovaSonicLLMService.Settings(
        model="amazon.nova-2-5-sonic",
        voice="tiffany",
        system_instruction="You are a helpful assistant.",
        temperature=0.5,
        max_tokens=2048,
        endpointing_sensitivity="MEDIUM",
    ),
)

With Function Calling

from pipecat.services.aws.nova_sonic.llm import AWSNovaSonicLLMService
from pipecat.services.llm_service import FunctionCallParams

llm = AWSNovaSonicLLMService(
    secret_access_key=os.getenv("AWS_SECRET_ACCESS_KEY"),
    access_key_id=os.getenv("AWS_ACCESS_KEY_ID"),
    region="us-east-1",
    settings=AWSNovaSonicLLMService.Settings(
        voice="matthew",
        system_instruction="You are a helpful assistant that can check the weather.",
    ),
    tools=tools,  # ToolsSchema instance
)

async def get_weather(params: FunctionCallParams):
    location = params.arguments.get("location", "unknown")
    await params.result_callback({"temperature": 72, "condition": "sunny", "location": location})

llm.register_function("get_weather", get_weather)

With Session Continuation

from pipecat.services.aws.nova_sonic.llm import AWSNovaSonicLLMService
from pipecat.services.aws.nova_sonic.session_continuation import SessionContinuationParams

llm = AWSNovaSonicLLMService(
    secret_access_key=os.getenv("AWS_SECRET_ACCESS_KEY"),
    access_key_id=os.getenv("AWS_ACCESS_KEY_ID"),
    region="us-east-1",
    settings=AWSNovaSonicLLMService.Settings(
        voice="tiffany",
        system_instruction="You are a helpful assistant.",
    ),
    # Session continuation is enabled by default. You can customize the behavior:
    session_continuation=SessionContinuationParams(
        enabled=True,
        transition_threshold_seconds=360,  # Start transition after 6 minutes
        audio_buffer_duration_seconds=3.0,  # Buffer 3 seconds of audio during transition
        audio_start_timeout_seconds=80.0,  # Wait up to 80s for the assistant to speak
    ),
)

# To disable session continuation:
# session_continuation=SessionContinuationParams(enabled=False)

Methods

reset_conversation()

async def reset_conversation()

Disconnects and reconnects, seeding the new session with the context’s messages. A trailing user message is sent for the model to answer. Before the service has received its first context, it only logs a warning.

Notes

Transcription frames: User speech transcription frames are always emitted upstream. Assistant text transcripts are delivered in real-time using speculative text events, providing text synchronized with audio output for responsive client UIs.
Connection resilience: If a connection error occurs while the service wants to stay connected, it automatically resets the conversation and reconnects. The same happens when the model calls a function that has no registered handler.
No user turn frames: The service emits no UserStartedSpeakingFrame or UserStoppedSpeakingFrame, so processors that depend on them (RTVI user speaking events, user idle and mute handling, turn recording) do nothing unless LLMUserAggregatorParams has a vad_analyzer.
System instruction precedence: The system_instruction from service settings takes precedence over an initial system message in the LLM context. A warning is logged when both are set. Tools provided in the LLM context take precedence over those provided at initialization time.
Audio format: Uses LPCM (Linear PCM) audio format for both input and output. Input defaults to 16kHz and output defaults to 24kHz. Input audio is sent without resampling, so input_sample_rate must match the input audio’s sample rate (audio_in_sample_rate, 16000 by default).

Get Started

Pipecat Server

Client SDKs

Pipecat Flows

Pipecat Cloud

CLI

Pipecat Context Hub

Overview

AWS Nova Sonic API Reference

Example Implementation

AWS Bedrock Documentation

AWS Console

Installation

Prerequisites

AWS Account Setup

Required Environment Variables

Key Features

Configuration

AWSNovaSonicLLMService

Settings

AudioConfig

SessionContinuationParams

Usage

Basic Setup

With Settings

With Function Calling

With Session Continuation

Methods

reset_conversation()

Notes

​Overview

AWS Nova Sonic API Reference

Example Implementation

AWS Bedrock Documentation

AWS Console

​Installation

​Prerequisites

​AWS Account Setup

​Required Environment Variables

​Key Features

​Configuration

​AWSNovaSonicLLMService

​Settings

​AudioConfig

​SessionContinuationParams

​Usage

​Basic Setup

​With Settings

​With Function Calling

​With Session Continuation

​Methods

​reset_conversation()

​Notes

Overview

Installation

Prerequisites

AWS Account Setup

Required Environment Variables

Key Features

Configuration

AWSNovaSonicLLMService

Settings

AudioConfig

SessionContinuationParams

Usage

Basic Setup

With Settings

With Function Calling

With Session Continuation

Methods

reset_conversation()

Notes