Overview
OLLamaLLMService
provides access to locally-run Ollama models through an OpenAI-compatible interface. It inherits from BaseOpenAILLMService
and allows you to run various open-source models locally while maintaining compatibility with OpenAI’s API format.
API Reference
Complete API documentation and method details
Ollama Docs
Official Ollama documentation and model library
Download Ollama
Download and setup instructions for Ollama
Installation
To use Ollama services, you need to install both Ollama and the Pipecat dependency:- Install Ollama on your system from ollama.com/download
- Install Pipecat dependency:
- Pull a model (first time only):
Ollama runs as a local service on port 11434. No API key required!
Frames
Input
OpenAILLMContextFrame
- Conversation context and historyLLMMessagesFrame
- Direct message listVisionImageRawFrame
- Images for vision modelsLLMUpdateSettingsFrame
- Runtime parameter updates
Output
LLMFullResponseStartFrame
/LLMFullResponseEndFrame
- Response boundariesLLMTextFrame
- Streamed completion chunksFunctionCallInProgressFrame
/FunctionCallResultFrame
- Function call lifecycleErrorFrame
- Connection or processing errors
Function Calling
Function Calling Guide
Learn how to implement function calling with standardized schemas, register
handlers, manage context properly, and control execution flow in your
conversational AI applications.
Context Management
Context Management Guide
Learn how to manage conversation context, handle message history, and
integrate context aggregators for consistent conversational experiences.
Usage Example
Metrics
Inherits all OpenAI metrics capabilities for local monitoring:- Time to First Byte (TTFB) - Local inference latency
- Processing Duration - Model execution time
- Token Usage - Local token counting (if supported by model)
Learn how to enable Metrics in your Pipeline.
Additional Notes
- Run models locally: Ollama allows you to run various open-source models on your own hardware, providing flexibility and control.
- OpenAI Compatibility: Full compatibility with OpenAI API features and parameters
- Privacy centric: All processing happens locally, ensuring data privacy and security.