NVIDIA NIM LLM

​

Overview

​

Installation

uv add "pipecat-ai[nvidia]"

​

Prerequisites

​

NVIDIA NIM Setup

1. NVIDIA Developer Account (cloud endpoint only): Sign up at NVIDIA Developer Portal
2. API Key (cloud endpoint only): Generate an NVIDIA API key for NIM cloud services
3. Model Selection: Choose from available NIM-hosted models
4. Local NIM Setup (optional): For local deployments, configure NIM on-premises and set the base_url to your local endpoint

​

Environment Variables

• NVIDIA_API_KEY: Your NVIDIA API key for authentication (required for the cloud endpoint)

​

Configuration

​

Settings

​

Usage

​

Basic Setup

import os
from pipecat.services.nvidia.llm import NvidiaLLMService

llm = NvidiaLLMService(
    api_key=os.getenv("NVIDIA_API_KEY"),
    settings=NvidiaLLMService.Settings(
        model="nvidia/nemotron-3-super-120b-a12b",
    ),
)

​

With Custom Settings

from pipecat.services.nvidia.llm import NvidiaLLMService

llm = NvidiaLLMService(
    api_key=os.getenv("NVIDIA_API_KEY"),
    settings=NvidiaLLMService.Settings(
        model="nvidia/nemotron-3-super-120b-a12b",
        temperature=0.7,
        top_p=0.9,
        max_completion_tokens=1024,
    ),
)

​

Notes

• Token reporting: Token usage is reported once per completion at the end of each request.
• Cloud vs. local deployment: NIM supports both cloud-hosted and on-premises deployments. For on-premises, override the base_url to point to your local NIM endpoint.
• Reasoning content: The service automatically detects and filters reasoning content from model responses, emitting it as LLMThought*Frame objects. This applies to:
  • Models with API-level reasoning separation (e.g., Nemotron Nano models) that include a reasoning_content or reasoning field
  • Models that begin their response with a <think>...</think> block (e.g., DeepSeek-R1, some Nemotron models)
  Reasoning frames are accessible to observers and logging but are not sent to TTS, keeping the spoken output clean while preserving visibility into the model’s thought process.