Overview
BackendLLMWorker runs any Pipecat LLM service as the backend of a conversational frontend: the agent that handles work needing tools or careful reasoning while the frontend keeps talking with the user. It extends LLMContextWorker, so the backend keeps a conversation of its own and calls its tools over as many steps as the work takes.
Two frontends drive it: LLMWithBackend, with any LLM service as the frontend, and OpenAILiveLLMService in client delegation. For a walkthrough, see LLM with a Backend.
Configuration
The backend LLM service. It runs at its provider’s default thinking for the
model unless its settings configure thinking.
The backend’s context, carrying its tools. A fresh empty context when
None.Unique name for this worker on the bus. Auto-generated when
None; give one
when the backend runs in another process, where the frontend addresses it by
name.Called with each
BackendOutput the model produces before
it is sent: return the output to send in its place, or None to send nothing.
Outputs sent with send_output() skip it unless that call
asks for it.How the model works with the frontend, appended to the LLM’s system
instruction. A custom override has to cover everything the default does: that
the frontend sends it requests, that the user hears what it marks with
SPOKEN_MARK and nothing else, and how to take requests that arrive while it
works.Parameters for the user aggregator. When
None, external turn strategies: the
backend has no audio to detect turns in.ValueError when the LLM service refuses the role. OpenAILiveLLMService does.
BackendOutput
One piece of output from the backend on its way to the frontend: whattransform_output receives and returns, and what send_output() sends.
BackendOutputTransform is the type of transform_output:
Properties
Inherits the properties ofLLMContextWorker.
working
True while the backend has work in hand: a request or model run outstanding, or a function call in flight.
Methods
send_output
transform_output unless apply_transform_output is True. With no frontend attached, the output is dropped with a warning.
Constants
Module constants ofpipecat.workers.llm.backend_llm_worker:
Notes
- What the user hears: A message the model starts with
SPOKEN_MARKis sent withprefers_spoken=True, mark removed, whether or not the same turn calls tools. Anything else it writes, reasoning summaries included, is sent withprefers_spoken=False: the frontend keeps it as a note and doesn’t say it. When the mark starts a later line, the text before that line is a note and the rest is spoken. - Requests while working: A request that arrives while the backend is working joins that work. The model reads it against what it has under way, and stops a long-running tool through the
cancel_<tool>tool thatcancellable_by_llm=Truegives it. - One frontend at a time: The worker serves one frontend. When that frontend lets go, such as when its pipeline ends, the backend’s work in progress is cancelled.