> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pipecat.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Gemini Live Vertex AI

> A real-time, multimodal conversational AI service powered by Google's Gemini via Vertex AI

## Overview

`GeminiLiveVertexLLMService` enables natural, real-time conversations with Google's Gemini model through Vertex AI. It provides built-in audio transcription, voice activity detection, and context management for creating interactive AI experiences with multimodal capabilities including audio, video, and text processing.

<Tip>
  Want to start building? Check out our [Gemini Live
  Guide](/pipecat/features/gemini-live) for general concepts, then follow the
  Vertex AI-specific setup below.
</Tip>

<CardGroup cols={2}>
  <Card title="Gemini Live Vertex API Reference" icon="code" href="https://reference-server.pipecat.ai/en/latest/api/pipecat.services.google.gemini_live.vertex.llm.html">
    Pipecat's API methods for Gemini Live Vertex AI integration
  </Card>

  <Card title="Example Implementation" icon="play" href="https://github.com/pipecat-ai/pipecat/blob/main/examples/realtime/realtime-gemini-live-vertex-function-calling.py">
    Complete Gemini Live Vertex AI function calling example
  </Card>

  <Card title="Vertex AI Gemini Documentation" icon="book" href="https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/multimodal-live">
    Official Vertex AI Gemini Live API documentation
  </Card>

  <Card title="Gemini Live Model Card" icon="book" href="https://docs.cloud.google.com/vertex-ai/generative-ai/docs/models/gemini/2-5-flash-live-api">
    Gemini Live available models
  </Card>
</CardGroup>

## Installation

To use Gemini Live Vertex AI services, install the required dependencies:

```bash theme={null}
uv add "pipecat-ai[google]"
```

## Prerequisites

### Google Cloud Setup

Before using Gemini Live Vertex AI services, you need:

1. **Google Cloud Project**: Set up a project in the [Google Cloud Console](https://console.cloud.google.com/)
2. **Vertex AI API**: Enable the Vertex AI API in your project
3. **Service Account**: Create a service account with `roles/aiplatform.user` and `roles/ml.developer` permissions
4. **Authentication**: Set up service account credentials or Application Default Credentials

### Required Environment Variables

* `GOOGLE_VERTEX_TEST_CREDENTIALS`: JSON string of service account credentials (optional if using ADC)
* `GOOGLE_CLOUD_PROJECT_ID`: Your Google Cloud project ID
* `GOOGLE_CLOUD_LOCATION`: Vertex AI region (e.g., "us-east4")

### Key Features

* **Enterprise Authentication**: Secure service account-based authentication
* **Multimodal Processing**: Handle audio, video, and text inputs simultaneously
* **Real-time Streaming**: Low-latency audio and video processing
* **Voice Activity Detection**: Automatic speech detection and turn management
* **Function Calling**: Advanced tool integration and API calling capabilities
* **Context Management**: Intelligent conversation history and system instruction handling

## Configuration

### GeminiLiveVertexLLMService

This service extends `GeminiLiveLLMService` with Vertex AI authentication. It accepts all the same parameters as the [Gemini Live](/api-reference/server/services/s2s/gemini-live) service, with these differences:

<ParamField path="credentials" type="str" default="None">
  JSON string of Google service account credentials. If not provided, falls back
  to `credentials_path` or Application Default Credentials (ADC).
</ParamField>

<ParamField path="credentials_path" type="str" default="None">
  Path to a service account JSON file. Used if `credentials` is not provided.
</ParamField>

<ParamField path="location" type="str" required>
  GCP region for the Vertex AI endpoint (e.g., `"us-east4"`).
</ParamField>

<ParamField path="project_id" type="str" required>
  Google Cloud project ID.
</ParamField>

<ParamField path="model" type="str" default="google/gemini-live-2.5-flash-native-audio" deprecated>
  Vertex AI model identifier to use.

  *Deprecated in v0.0.105. Use `settings=GeminiLiveVertexLLMService.Settings(model=...)` instead.*
</ParamField>

<ParamField path="voice_id" type="str" default="Charon" deprecated>
  TTS voice identifier for audio responses.

  *Deprecated in v0.0.105. Use `settings=GeminiLiveVertexLLMService.Settings(voice=...)` instead.*
</ParamField>

<ParamField path="system_instruction" type="str" default="None">
  System prompt for the model. Can also be provided via the LLM context.
</ParamField>

<ParamField path="tools" type="List[dict] | ToolsSchema" default="None">
  Tools/functions available to the model. Can also be provided via the LLM
  context.
</ParamField>

<ParamField path="params" type="InputParams" default="InputParams()" deprecated>
  Runtime-configurable generation and session settings. See the [Gemini Live
  InputParams](/api-reference/server/services/s2s/gemini-live#inputparams) for details.

  *Deprecated in v0.0.105. Use `settings=GeminiLiveVertexLLMService.Settings(...)` instead.*
</ParamField>

<ParamField path="settings" type="GeminiLiveVertexLLMService.Settings" default="None">
  Runtime-configurable settings. See the [Gemini Live
  Settings](/api-reference/server/services/s2s/gemini-live#settings) for the
  full reference.
</ParamField>

<ParamField path="start_audio_paused" type="bool" default="False">
  Whether to start with audio input paused.
</ParamField>

<ParamField path="start_video_paused" type="bool" default="False">
  Whether to start with video input paused.
</ParamField>

<ParamField path="inference_on_context_initialization" type="bool" default="True">
  Whether to generate a response when context is first set. Set to `False` to
  wait for user input before the model responds.
</ParamField>

<ParamField path="http_options" type="HttpOptions" default="None">
  HTTP options for the Google API client.
</ParamField>

### Settings

The Vertex AI variant uses the same Settings as the base Gemini Live service. See [Gemini Live Settings](/api-reference/server/services/s2s/gemini-live#settings) for the full reference.

## Usage

### Basic Setup with Service Account Credentials

```python theme={null}
import os
from pipecat.services.google.gemini_live import GeminiLiveVertexLLMService

llm = GeminiLiveVertexLLMService(
    credentials=os.getenv("GOOGLE_VERTEX_TEST_CREDENTIALS"),
    project_id=os.getenv("GOOGLE_CLOUD_PROJECT_ID"),
    location=os.getenv("GOOGLE_CLOUD_LOCATION"),
    settings=GeminiLiveVertexLLMService.Settings(
        voice="Charon",
        system_instruction="You are a helpful assistant.",
    ),
)
```

### With Credentials File

```python theme={null}
llm = GeminiLiveVertexLLMService(
    credentials_path="/path/to/service-account.json",
    project_id="my-gcp-project",
    location="us-east4",
    settings=GeminiLiveVertexLLMService.Settings(
        voice="Puck",
        system_instruction="You are a helpful assistant.",
    ),
)
```

### Using Application Default Credentials (ADC)

```python theme={null}
# When running on GCP or with gcloud auth application-default login
llm = GeminiLiveVertexLLMService(
    project_id="my-gcp-project",
    location="us-east4",
    settings=GeminiLiveVertexLLMService.Settings(
        system_instruction="You are a helpful assistant.",
    ),
)
```

### With Settings

```python theme={null}
from pipecat.services.google.gemini_live import GeminiVADParams

llm = GeminiLiveVertexLLMService(
    credentials=os.getenv("GOOGLE_VERTEX_TEST_CREDENTIALS"),
    project_id=os.getenv("GOOGLE_CLOUD_PROJECT_ID"),
    location="us-east4",
    settings=GeminiLiveVertexLLMService.Settings(
        model="google/gemini-live-2.5-flash-native-audio",
        voice="Charon",
        system_instruction="You are a helpful assistant.",
        temperature=0.7,
        max_tokens=2048,
        vad=GeminiVADParams(
            silence_duration_ms=500,
        ),
    ),
)
```

<Tip>
  The `InputParams` / `params=` pattern is deprecated as of v0.0.105. Use
  `Settings` / `settings=` instead. See the [Service Settings
  guide](/pipecat/fundamentals/service-settings) for migration details.
</Tip>

## Notes

* **No `api_key` parameter**: Unlike the base `GeminiLiveLLMService`, Vertex AI uses service account credentials or ADC for authentication. Passing `api_key` will raise a `ValueError`.
* **Authentication priority**: The service tries credentials in this order: (1) `credentials` JSON string, (2) `credentials_path` file, (3) Application Default Credentials (ADC).
* **File API not supported**: The Gemini File API is not available through Vertex AI. Use Google Cloud Storage for file handling instead.
* **Model naming**: Vertex AI uses different model identifiers (e.g., `"google/gemini-live-2.5-flash-native-audio"`) compared to the Google AI variant.
* **All other features** (VAD, context compression, thinking, function calling, etc.) work identically to the base [Gemini Live](/api-reference/server/services/s2s/gemini-live) service.