Overview

GoogleLLMService provides integration with Google’s Gemini models, supporting streaming responses, function calling, and multimodal inputs. It includes specialized context handling for Google’s message format while maintaining compatibility with OpenAI-style contexts.

Installation

To use GoogleLLMService, install the required dependencies:

pip install pipecat-ai[google]

You’ll also need to set up your Google API key as an environment variable: GOOGLE_API_KEY

Configuration

Constructor Parameters

api_key
str
required

Google API key

model
str
default: "gemini-1.5-flash-latest"

Model identifier

params
InputParams

Model configuration parameters

Input Parameters

extra
Optional[Dict[str, Any]]
default: "{}"

Additional parameters to pass to the model

max_tokens
Optional[int]
default: "4096"

Maximum number of tokens to generate. Must be greater than or equal to 1

temperature
Optional[float]
default: "None"

Controls randomness in the output. Range: [0.0, 2.0]

top_k
Optional[int]
default: "None"

Controls diversity via nucleus sampling. Must be greater than or equal to 0

top_p
Optional[float]
default: "None"

Controls diversity via nucleus sampling. Range: [0.0, 1.0]

Input Frames

OpenAILLMContextFrame
Frame

Contains conversation context

LLMMessagesFrame
Frame

Contains conversation messages

VisionImageRawFrame
Frame

Contains image for vision processing

LLMUpdateSettingsFrame
Frame

Updates model settings

Output Frames

TextFrame
Frame

Contains generated text

LLMFullResponseStartFrame
Frame

Signals start of response

LLMFullResponseEndFrame
Frame

Signals end of response

Context Management

The Google service uses specialized context management to handle conversations and message formatting. This includes managing the conversation history, system prompts, function calls, and converting between OpenAI and Google message formats.

GoogleLLMContext

The base context manager for Google conversations:

context = GoogleLLMContext(
    messages=[],  # Conversation history
    tools=[],     # Available function calling tools
    system="You are a helpful assistant"  # System prompt
)

Context Aggregators

Context aggregators handle message format conversion and management. The service provides a method to create paired aggregators:

create_context_aggregator
static method

Creates user and assistant aggregators for handling message formatting.

@staticmethod
def create_context_aggregator(
    context: OpenAILLMContext,
    *,
    assistant_expect_stripped_words: bool = True
) -> GoogleContextAggregatorPair

Parameters

context
OpenAILLMContext
required

The context object containing conversation history and settings

assistant_expect_stripped_words
bool
default: "True"

Controls text preprocessing for assistant responses

Usage Example


# 1. Create the context
context = GoogleLLMContext(
    messages=[],
    system="You are a helpful assistant"
)

# 2. Create aggregators for message handling
aggregators = GoogleLLMService.create_context_aggregator(context)

# 3. Access individual aggregators
user_aggregator = aggregators.user()      # Handles user message formatting
assistant_aggregator = aggregators.assistant()  # Handles assistant responses

# 4. Use in a pipeline
pipeline = Pipeline([
    user_aggregator,
    llm_service,
    assistant_aggregator
])

The context management system ensures proper message formatting and history tracking throughout the conversation while handling the conversion between OpenAI and Google message formats automatically.

Methods

See the LLM base class methods for additional functionality.

Usage Examples

Basic Usage

# Configure service
llm_service = GoogleLLMService(
    api_key="your-api-key",
    model="gemini-1.5-flash-latest",
    params=GoogleLLMService.InputParams(
        temperature=0.7,
        max_tokens=1000
    )
)

# Create pipeline
pipeline = Pipeline([
    context_manager,
    llm_service,
    response_handler
])

With Function Calling

# Configure function calling
context = GoogleLLMContext()
context.add_tool({
    "function_declarations": [{
        "name": "get_weather",
        "description": "Get weather information",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string"}
            }
        }
    }]
})

Frame Flow

Metrics Support

The service collects various metrics:

  • Token usage (prompt and completion)
  • Processing time
  • Time to first byte (TTFB)

Notes

  • Supports streaming responses
  • Handles function calling
  • Provides OpenAI compatibility
  • Manages conversation context
  • Supports vision inputs
  • Includes metrics collection
  • Thread-safe processing