Ollama LLM

​

Overview

​

Installation

1. Install Ollama on your system from ollama.com/download
2. Install Pipecat dependency:

uv add pipecat-ai

3. Pull a model (first time only):

ollama pull llama2

​

Prerequisites

​

Ollama Local Setup

1. Ollama Installation: Download and install Ollama from ollama.com
2. Model Selection: Pull your desired models (llama2, mistral, codellama, etc.)
3. Local Service: Ensure Ollama service is running (default port 11434)
4. Hardware: Sufficient RAM and storage for your chosen models

​

Configuration

• No API Keys Required: Ollama runs entirely locally
• Model Management: Use ollama pull <model> to download models
• Service URL: Default is http://localhost:11434 (configurable)

​

Configuration

​

Settings

​

Usage

​

Basic Setup

from pipecat.services.ollama.llm import OLLamaLLMService

llm = OLLamaLLMService(
    settings=OLLamaLLMService.Settings(model="llama2"),
)

​

With Custom Model and URL

from pipecat.services.ollama.llm import OLLamaLLMService

llm = OLLamaLLMService(
    settings=OLLamaLLMService.Settings(model="mistral"),
    base_url="http://localhost:11434/v1",
)

​

With Custom Settings

from pipecat.services.ollama.llm import OLLamaLLMService

llm = OLLamaLLMService(
    base_url="http://localhost:11434/v1",
    settings=OLLamaLLMService.Settings(
        model="mistral",
        temperature=0.7,
    ),
)

​

Notes

• No API key is required for a local Ollama. The service passes the placeholder key "ollama" unless you set api_key, which you only need for a server behind an authenticating proxy.
• The Ollama service must be running locally before starting your pipeline. Start it with ollama serve if it is not already running.
• Model capabilities (function calling, vision, etc.) depend on the specific model you pull. Check the Ollama model library for details.