Gemini Live Vertex AI

​

Overview

​

Installation

uv add "pipecat-ai[google]"

​

Prerequisites

​

Google Cloud Setup

1. Google Cloud Project: Set up a project in the Google Cloud Console
2. Vertex AI API: Enable the Vertex AI API in your project
3. Service Account: Create a service account with roles/aiplatform.user and roles/ml.developer permissions
4. Authentication: Set up service account credentials or Application Default Credentials

​

Required Environment Variables

• GOOGLE_VERTEX_TEST_CREDENTIALS: JSON string of service account credentials (optional if using ADC)
• GOOGLE_CLOUD_PROJECT_ID: Your Google Cloud project ID
• GOOGLE_CLOUD_LOCATION: Vertex AI region (e.g., “us-east4”)

​

Key Features

• Enterprise Authentication: Secure service account-based authentication
• Multimodal Processing: Handle audio, video, and text inputs simultaneously
• Real-time Streaming: Low-latency audio and video processing
• Voice Activity Detection: Automatic speech detection and turn management
• Function Calling: Advanced tool integration and API calling capabilities
• Context Management: Intelligent conversation history and system instruction handling

​

Configuration

​

GeminiLiveVertexLLMService

​

Settings

​

Usage

​

Basic Setup with Service Account Credentials

import os
from pipecat.services.google.gemini_live.vertex.llm import GeminiLiveVertexLLMService

llm = GeminiLiveVertexLLMService(
    credentials=os.getenv("GOOGLE_VERTEX_TEST_CREDENTIALS"),
    project_id=os.getenv("GOOGLE_CLOUD_PROJECT_ID"),
    location=os.getenv("GOOGLE_CLOUD_LOCATION"),
    settings=GeminiLiveVertexLLMService.Settings(
        voice="Charon",
        system_instruction="You are a helpful assistant.",
    ),
)

​

With Credentials File

llm = GeminiLiveVertexLLMService(
    credentials_path="/path/to/service-account.json",
    project_id="my-gcp-project",
    location="us-east4",
    settings=GeminiLiveVertexLLMService.Settings(
        voice="Puck",
        system_instruction="You are a helpful assistant.",
    ),
)

​

Using Application Default Credentials (ADC)

# When running on GCP or with gcloud auth application-default login
llm = GeminiLiveVertexLLMService(
    project_id="my-gcp-project",
    location="us-east4",
    settings=GeminiLiveVertexLLMService.Settings(
        system_instruction="You are a helpful assistant.",
    ),
)

​

With Settings

from pipecat.services.google.gemini_live.llm import GeminiVADParams

llm = GeminiLiveVertexLLMService(
    credentials=os.getenv("GOOGLE_VERTEX_TEST_CREDENTIALS"),
    project_id=os.getenv("GOOGLE_CLOUD_PROJECT_ID"),
    location="us-east4",
    settings=GeminiLiveVertexLLMService.Settings(
        model="google/gemini-3.8-live",
        voice="Charon",
        system_instruction="You are a helpful assistant.",
        temperature=0.7,
        max_tokens=2048,
        vad=GeminiVADParams(
            silence_duration_ms=500,
        ),
    ),
)

​

Methods

​

Notes

• No api_key parameter: Unlike the base GeminiLiveLLMService, Vertex AI uses service account credentials or ADC for authentication. Passing api_key will raise a ValueError.
• Authentication priority: The service tries credentials in this order: (1) credentials JSON string, (2) credentials_path file, (3) Application Default Credentials (ADC). If none of these yields credentials, the constructor raises ValueError.
• File API not supported: The Gemini File API is not available through Vertex AI. Use Google Cloud Storage for file handling instead. Accessing file_api raises NotImplementedError.
• Model naming: Vertex AI uses different model identifiers (e.g., "google/gemini-3.8-live") compared to the Google AI variant.
• Async tool limitation: Vertex AI’s Gemini Live endpoint does not support NON_BLOCKING tool calls. Functions registered with cancel_on_interruption=False fall back to synchronous behavior (the conversation pauses while the tool runs), and the service logs an error and pushes a non-fatal ErrorFrame the first time one is used. Use cancel_on_interruption=True (the default) or use a non-realtime LLM service if your tool requires async semantics.
• All other features (VAD, context compression, thinking, function calling, etc.) work identically to the base Gemini Live service.