​
Overview

GrokLLMService provides access to Grok’s language models through an OpenAI-compatible interface. It inherits from OpenAILLMService and supports streaming responses, function calling, and context management.

API Reference

Complete API documentation and method details

Grok Docs

Official Grok API documentation and features

Example Code

Working example with function calling

​
Installation

To use GrokLLMService, install the required dependencies:

pip install "pipecat-ai[grok]"

You’ll also need to set up your Grok API key as an environment variable: GROK_API_KEY.

Get your API key from X.AI Console.

​
Frames

​
Input

  • OpenAILLMContextFrame - Conversation context and history
  • LLMMessagesFrame - Direct message list
  • VisionImageRawFrame - Images for vision processing
  • LLMUpdateSettingsFrame - Runtime parameter updates

​
Output

  • LLMFullResponseStartFrame / LLMFullResponseEndFrame - Response boundaries
  • LLMTextFrame - Streamed completion chunks
  • FunctionCallInProgressFrame / FunctionCallResultFrame - Function call lifecycle
  • ErrorFrame - API or processing errors

​
Function Calling

Function Calling Guide

Learn how to implement function calling with standardized schemas, register handlers, manage context properly, and control execution flow in your conversational AI applications.

​
Context Management

Context Management Guide

Learn how to manage conversation context, handle message history, and integrate context aggregators for consistent conversational experiences.

​
Usage Example

import os
from pipecat.services.grok.llm import GrokLLMService
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.adapters.schemas.function_schema import FunctionSchema
from pipecat.adapters.schemas.tools_schema import ToolsSchema

# Configure Grok service
llm = GrokLLMService(
    api_key=os.getenv("GROK_API_KEY"),
    model="grok-3-beta",
    params=GrokLLMService.InputParams(
        temperature=0.8,  # Higher for creative responses
        max_tokens=1000
    )
)

# Define function for tool calling
weather_function = FunctionSchema(
    name="get_current_weather",
    description="Get current weather information",
    properties={
        "location": {
            "type": "string",
            "description": "City and state, e.g. San Francisco, CA"
        },
        "format": {
            "type": "string",
            "enum": ["celsius", "fahrenheit"],
            "description": "Temperature unit to use"
        }
    },
    required=["location", "format"]
)

tools = ToolsSchema(standard_tools=[weather_function])

# Create context optimized for voice interaction
context = OpenAILLMContext(
    messages=[
        {
            "role": "system",
            "content": """You are a helpful and creative assistant in a voice conversation.
            Your output will be converted to audio, so avoid special characters.
            Respond in an engaging and helpful way while being succinct."""
        }
    ],
    tools=tools
)

# Create context aggregators
context_aggregator = llm.create_context_aggregator(context)

# Register function handler
async def fetch_weather(params):
    location = params.arguments["location"]
    await params.result_callback({"conditions": "sunny", "temperature": "75°F"})

llm.register_function("get_current_weather", fetch_weather)

# Use in pipeline
pipeline = Pipeline([
    transport.input(),
    stt,
    context_aggregator.user(),
    llm,
    tts,
    transport.output(),
    context_aggregator.assistant()
])

​
Metrics

Inherits all OpenAI metrics capabilities with specialized token tracking:

  • Time to First Byte (TTFB) - Response latency measurement
  • Processing Duration - Total request processing time
  • Token Usage - Accumulated prompt tokens, completion tokens, and totals

Grok uses incremental token reporting, so metrics are accumulated and reported at the end of each response.

Enable with:

task = PipelineTask(
    pipeline,
    params=PipelineParams(
        enable_metrics=True,
        enable_usage_metrics=True
    )
)

​
Additional Notes

  • OpenAI Compatibility: Full compatibility with OpenAI API features and parameters
  • Real-time Information: Access to current events and up-to-date information
  • Vision Capabilities: Image understanding and analysis with grok-2-vision model