Google Vertex AI

​

Overview

GoogleVertexLLMService provides access to Google’s language models through Vertex AI while maintaining an OpenAI-compatible interface. It inherits from OpenAILLMService and supports all the features of the OpenAI interface while connecting to Google’s AI services.

​

Installation

To use GoogleVertexLLMService, install the required dependencies:

pip install "pipecat-ai[google]"

You’ll also need to set up Google Cloud credentials. You can either:

• Set the GOOGLE_APPLICATION_CREDENTIALS environment variable pointing to your service account JSON file
• Provide credentials directly to the service constructor

​

Configuration

​

Constructor Parameters

​

Input Parameters

Extends the OpenAI input parameters with Vertex AI specific options:

Also inherits all OpenAI-compatible parameters:

​

Usage Example

from pipecat.services.google.llm_vertex import GoogleVertexLLMService
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.task import PipelineParams, PipelineTask

# Configure service
llm = GoogleVertexLLMService(
    credentials_path="/path/to/service-account.json",
    model="google/gemini-2.0-flash-001",
    params=GoogleVertexLLMService.InputParams(
        project_id="your-google-cloud-project-id",
        location="us-east4"
    )
)

# Create context with system message
context = OpenAILLMContext(
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant in a voice conversation. Keep responses concise."
        }
    ]
)

# Create context aggregator for message handling
context_aggregator = llm.create_context_aggregator(context)

# Set up pipeline
pipeline = Pipeline([
    transport.input(),
    context_aggregator.user(),
    llm,
    tts,
    transport.output(),
    context_aggregator.assistant()
])

# Create and configure task
task = PipelineTask(
    pipeline,
    params=PipelineParams(
        allow_interruptions=True,
        enable_metrics=True,
        enable_usage_metrics=True,
    ),
)

​

Authentication

The service supports multiple authentication methods:

1. Direct credentials string - Pass the JSON credentials as a string to the constructor
2. Credentials file path - Provide a path to the service account JSON file
3. Environment variable - Set GOOGLE_APPLICATION_CREDENTIALS to the path of your service account file

The service automatically handles token refresh, with tokens having a 1-hour lifetime.

​

Methods

See the LLM base class methods for additional functionality.

​

Function Calling

This service supports function calling (also known as tool calling) through the OpenAI-compatible interface, which allows the LLM to request information from external services and APIs.

​

Available Models

​

Frame Flow

Inherits the OpenAI LLM Service frame flow:

​

Metrics Support

The service collects standard LLM metrics:

• Token usage (prompt and completion)
• Processing duration
• Time to First Byte (TTFB)
• Function call metrics

​

Notes

• Uses Google Cloud’s Vertex AI API
• Maintains OpenAI-compatible interface
• Supports streaming responses
• Handles function calling
• Manages conversation context
• Includes token usage tracking
• Thread-safe processing
• Automatic token refresh
• Requires Google Cloud project setup