AWS Nova Sonic

The AWSNovaSonicLLMService enables natural, real-time conversations with AWS Nova Sonic. It provides built-in audio transcription, voice activity detection, and context management for creating interactive AI experiences. It provides:

Real-time Interaction

Stream audio in real-time with low latency response times

Speech Processing

Built-in speech-to-text and text-to-speech capabilities with multiple voice options

Voice Activity Detection

Automatic detection of speech start/stop for natural conversations

Context Management

Intelligent handling of conversation history and system instructions

Installation

To use AWSNovaSonicLLMService, install the required dependencies:

pip install "pipecat-ai[aws-nova-sonic]"

We recommend setting up your AWS credentials as environment variables, as you’ll need them to initialize AWSNovaSonicLLMService:

AWS_SECRET_ACCESS_KEY
AWS_ACCESS_KEY_ID
AWS_REGION

Basic Usage

Here’s a simple example of setting up a conversational AI bot with AWS Nova Sonic:

from pipecat.services.aws_nova_sonic.aws import AWSNovaSonicLLMService

llm = AWSNovaSonicLLMService(
    secret_access_key=os.getenv("AWS_SECRET_ACCESS_KEY"),
    access_key_id=os.getenv("AWS_ACCESS_KEY_ID"),
    region=os.getenv("AWS_REGION")
    voice_id="tiffany",                    # Voices: matthew, tiffany, amy
)

Configuration

Constructor Parameters

secret_access_key

str

required

Your AWS secret access key

access_key_id

str

required

Your AWS access key ID

region

str

required

Specify the AWS region for the service (e.g., "us-east-1"). Note that the service may not be available in all AWS regions: check the AWS Bedrock User Guide’s support table.

model

str

default:"amazon.nova-sonic-v1:0"

AWS Nova Sonic model to use. Note that "amazon.nova-sonic-v1:0" is the only supported model as of 2025-05-08.

voice_id

str

default:"matthew"

Voice for text-to-speech (options: "matthew", "tiffany", "amy")

params

Params

Configuration for model parameters

system_instruction

str

High-level instructions that guide the model’s behavior. Note that more commonly these instructions will be included as part of the context provided to kick off the conversation.

tools

ToolsSchema

List of function definitions for tool/function calling. Note that more commonly tools will be included as part of the context provided to kick off the conversation.

send_transcription_frames

bool

default:"True"

Whether to emit transcription frames

Model Parameters

The Params object configures the behavior of the AWS Nova Sonic model.

It is strongly recommended to stick with default values (most easily by omitting params when constructing AWSNovaSonicLLMService) unless you have a good understanding of the parameters and their impact. Deviating from the defaults may lead to unexpected behavior.

temperature

float

default:"0.7"

Controls randomness in responses. Range: 0.0 to 2.0

max_tokens

int

default:"1024"

Maximum number of tokens to generate

top_p

float

default:"0.9"

Cumulative probability cutoff for token selection. Range: 0.0 to 1.0

input_sample_rate

int

default:"16000"

Sample rate for input audio

output_sample_rate

int

default:"24000"

Sample rate for output audio

input_sample_size

int

default:"16"

Bit depth for input audio

input_channel_count

int

default:"1"

Number of channels for input audio

output_sample_size

int

default:"16"

Bit depth for output audio

output_channel_count

int

default:"1"

Number of channels for output audio

Frame Types

Input Frames

InputAudioRawFrame

Frame

Raw audio data for speech input

OpenAILLMContextFrame

Frame

Contains conversation context

BotStoppedSpeakingFrame

Frame

Signals the bot has stopped speaking

Output Frames

TTSAudioRawFrame

Frame

Generated speech audio

LLMFullResponseStartFrame

Frame

Signals the start of a response from the LLM

LLMFullResponseEndFrame

Frame

Signals the end of a response from the LLM

TTSStartedFrame

Frame

Signals start of speech synthesis (coincides with the start of the LLM response, as this is a speech-to-speech model)

TTSStoppedFrame

Frame

Signals end of speech synthesis (coincides with the end of the LLM response, as this is a speech-to-speech model)

LLMTextFrame

Frame

Generated text responses from the LLM

TTSTextFrame

Frame

Generated text responses

TranscriptionFrame

Frame

Speech transcriptions. Only output if send_transcription_frames is True.

Function Calling

This service supports function calling (also known as tool calling) which allows the LLM to request information from external services and APIs. For example, you can enable your bot to:

Check current weather conditions
Query databases
Access external APIs
Perform custom actions

See the Function Calling guide for:

Detailed implementation instructions
Provider-specific function definitions
Handler registration examples
Control over function call behavior
Complete usage examples

Next Steps

Examples

Foundational Example Basic implementation showing core features
Persistent Content Example Implementation showing saving and loading conversation history

On this page

Installation
Basic Usage
Configuration
Constructor Parameters
Model Parameters
Frame Types
Input Frames
Output Frames
Function Calling
Next Steps
Examples

Real-time Interaction

Stream audio in real-time with low latency response times

Speech Processing

Built-in speech-to-text and text-to-speech capabilities with multiple voice options

Voice Activity Detection

Automatic detection of speech start/stop for natural conversations

Context Management

Intelligent handling of conversation history and system instructions

Installation

To use AWSNovaSonicLLMService, install the required dependencies:

pip install "pipecat-ai[aws-nova-sonic]"

We recommend setting up your AWS credentials as environment variables, as you’ll need them to initialize AWSNovaSonicLLMService:

AWS_SECRET_ACCESS_KEY
AWS_ACCESS_KEY_ID
AWS_REGION

Basic Usage

Here’s a simple example of setting up a conversational AI bot with AWS Nova Sonic:

from pipecat.services.aws_nova_sonic.aws import AWSNovaSonicLLMService

llm = AWSNovaSonicLLMService(
    secret_access_key=os.getenv("AWS_SECRET_ACCESS_KEY"),
    access_key_id=os.getenv("AWS_ACCESS_KEY_ID"),
    region=os.getenv("AWS_REGION")
    voice_id="tiffany",                    # Voices: matthew, tiffany, amy
)

Configuration

Constructor Parameters

secret_access_key

str

required

Your AWS secret access key

access_key_id

str

required

Your AWS access key ID

region

str

required

Specify the AWS region for the service (e.g., "us-east-1"). Note that the service may not be available in all AWS regions: check the AWS Bedrock User Guide’s support table.

model

str

default:"amazon.nova-sonic-v1:0"

AWS Nova Sonic model to use. Note that "amazon.nova-sonic-v1:0" is the only supported model as of 2025-05-08.

voice_id

str

default:"matthew"

Voice for text-to-speech (options: "matthew", "tiffany", "amy")

params

Params

Configuration for model parameters

system_instruction

str

High-level instructions that guide the model’s behavior. Note that more commonly these instructions will be included as part of the context provided to kick off the conversation.

tools

ToolsSchema

List of function definitions for tool/function calling. Note that more commonly tools will be included as part of the context provided to kick off the conversation.

send_transcription_frames

bool

default:"True"

Whether to emit transcription frames

Model Parameters

The Params object configures the behavior of the AWS Nova Sonic model.

temperature

float

default:"0.7"

Controls randomness in responses. Range: 0.0 to 2.0

max_tokens

int

default:"1024"

Maximum number of tokens to generate

top_p

float

default:"0.9"

Cumulative probability cutoff for token selection. Range: 0.0 to 1.0

input_sample_rate

int

default:"16000"

Sample rate for input audio

output_sample_rate

int

default:"24000"

Sample rate for output audio

input_sample_size

int

default:"16"

Bit depth for input audio

input_channel_count

int

default:"1"

Number of channels for input audio

output_sample_size

int

default:"16"

Bit depth for output audio

output_channel_count

int

default:"1"

Number of channels for output audio

Frame Types

Input Frames

InputAudioRawFrame

Frame

Raw audio data for speech input

OpenAILLMContextFrame

Frame

Contains conversation context

BotStoppedSpeakingFrame

Frame

Signals the bot has stopped speaking

Output Frames

TTSAudioRawFrame

Frame

Generated speech audio

LLMFullResponseStartFrame

Frame

Signals the start of a response from the LLM

LLMFullResponseEndFrame

Frame

Signals the end of a response from the LLM

TTSStartedFrame

Frame

Signals start of speech synthesis (coincides with the start of the LLM response, as this is a speech-to-speech model)

TTSStoppedFrame

Frame

Signals end of speech synthesis (coincides with the end of the LLM response, as this is a speech-to-speech model)

LLMTextFrame

Frame

Generated text responses from the LLM

TTSTextFrame

Frame

Generated text responses

TranscriptionFrame

Frame

Speech transcriptions. Only output if send_transcription_frames is True.

Function Calling

This service supports function calling (also known as tool calling) which allows the LLM to request information from external services and APIs. For example, you can enable your bot to:

Check current weather conditions
Query databases
Access external APIs
Perform custom actions

See the Function Calling guide for:

Detailed implementation instructions
Provider-specific function definitions
Handler registration examples
Control over function call behavior
Complete usage examples

Next Steps

Examples

Foundational Example Basic implementation showing core features
Persistent Content Example Implementation showing saving and loading conversation history

On this page

Installation
Basic Usage
Configuration
Constructor Parameters
Model Parameters
Frame Types
Input Frames
Output Frames
Function Calling
Next Steps
Examples

Real-time Interaction

Speech Processing

Voice Activity Detection

Context Management

Installation

Basic Usage

Configuration

Constructor Parameters

Model Parameters

Frame Types

Input Frames

Output Frames

Function Calling

Next Steps

Examples

Javascript SDK

React SDK

React Native SDK

iOS SDK

Android SDK

C++ SDK

AWS Nova Sonic

Real-time Interaction

Speech Processing

Voice Activity Detection

Context Management

Installation

Basic Usage

Configuration

Constructor Parameters

Model Parameters

Frame Types

Input Frames

Output Frames

Function Calling

Next Steps

Examples

Real-time Interaction

Speech Processing

Voice Activity Detection

Context Management

​Installation

​Basic Usage

​Configuration

​Constructor Parameters

​Model Parameters

​Frame Types

​Input Frames

​Output Frames

​Function Calling

​Next Steps

​Examples

Javascript SDK

React SDK

React Native SDK

iOS SDK

Android SDK

C++ SDK

Real-time Interaction

Speech Processing

Voice Activity Detection

Context Management

​Installation

​Basic Usage

​Configuration

​Constructor Parameters

​Model Parameters

​Frame Types

​Input Frames

​Output Frames

​Function Calling

​Next Steps

​Examples

Installation

Basic Usage

Configuration

Constructor Parameters

Model Parameters

Frame Types

Input Frames

Output Frames

Function Calling

Next Steps

Examples

Installation

Basic Usage

Configuration

Constructor Parameters

Model Parameters

Frame Types

Input Frames

Output Frames

Function Calling

Next Steps

Examples