> ## Documentation Index > Fetch the complete documentation index at: https://docs.pipecat.ai/llms.txt > Use this file to discover all available pages before exploring further. # LLM with a Backend > Pair a conversational frontend LLM with a backend LLM that handles work needing tools or careful reasoning, using LLMWithBackend. > [!WARNING] > > `LLMWithBackend` is in alpha: its API is likely to change between releases. `LLMWithBackend` pairs a **frontend** LLM, which keeps the conversation with the user fluent, with a **backend** LLM that handles work needing tools or careful reasoning. It goes where the LLM goes in your pipeline and runs the backend as an agent of its own. The frontend hands work to the backend with a `delegate` tool and keeps talking with the user, and what the backend has to say comes back into the frontend's conversation for it to relay to the user. It's a good fit for a voice bot whose work may involve nontrivial tool calls, several tool calls, or real reasoning: the user keeps talking with the frontend while the backend works. ## Basic Usage Define the backend's [tools](https://docs.pipecat.ai/pipecat/learn/function-calling.md) as you would for any LLM: ```python theme={null} from pipecat.services.llm_service import FunctionCallParams async def run_tests(params: FunctionCallParams, path: str | None = None): """Run the test suite, or one test file. Args: path: A test file to run alone; the whole suite when omitted. """ result = await test_runner.run(path) await params.result_callback(result) ``` Give the frontend a service of its own and the backend a [`BackendLLMWorker`](https://docs.pipecat.ai/api-reference/server/workers/backend-llm-worker) holding the backend's LLM and tools: ```python theme={null} import os from pipecat.pipeline.llm_with_backend import LLMWithBackend from pipecat.processors.aggregators.llm_context import LLMContext from pipecat.services.anthropic.llm import AnthropicLLMService from pipecat.services.openai.responses.llm import OpenAIResponsesLLMService from pipecat.workers.llm import BackendLLMWorker llm = LLMWithBackend( # By default, a frontend runs like any LLM in a pipeline: services that set a # reasoning default, as this one does, run a reasoning model at its lowest # reasoning effort, for fast replies. frontend=OpenAIResponsesLLMService( api_key=os.getenv("OPENAI_API_KEY"), settings=OpenAIResponsesLLMService.Settings(system_instruction=FRONTEND_INSTRUCTIONS), ), # By default, a backend runs its model at the provider's default reasoning # effort, since it does the reasoning the frontend hands off. backend=BackendLLMWorker( llm=AnthropicLLMService( api_key=os.getenv("ANTHROPIC_API_KEY"), settings=AnthropicLLMService.Settings( model="claude-opus-5-5", system_instruction=BACKEND_INSTRUCTIONS, ), ), context=LLMContext(tools=[search_codebase, read_file, apply_patch, run_tests]), ), ) ``` Put `llm` where the LLM goes in the pipeline: ```python theme={null} pipeline = Pipeline([ transport.input(), stt, aggregators.user(), llm, tts, transport.output(), aggregators.assistant(), ]) ``` The frontend needs no tools of its own: `LLMWithBackend` gives it the `delegate` tool. `LLMWithBackend` is a pipeline, not an LLM service, so functions, settings updates and event handlers go on `llm.frontend`. The backend is added to your pipeline worker when the pipeline starts, so there is nothing else to register. ## Writing the Prompts Each model gets a system instruction from you. The frontend's says who the assistant is and, in plain words, what work it can take on: ```python theme={null} FRONTEND_INSTRUCTIONS = """You are Pip, the voice assistant of the Acme engineering team. Your responses are spoken aloud, so keep them to one or two natural sentences. You can make code changes and run the tests, research the team's internal docs, and check CI and the open pull requests; that work takes a moment.""" ``` The backend's says what it does with its tools and what the user should be told: ```python theme={null} BACKEND_INSTRUCTIONS = """You do the engineering work for the Acme team's assistant. When asked for a code change, keep going until the tests pass, then tell the user what you changed and the test result. Keep what you tell them short and concrete.""" ``` Pipecat appends a **pairing instruction** to each, the text that tells a model how it works with the other: the frontend's says when and how to delegate and what to do with the backend's messages, and the backend's says how to take requests and how to mark what the user should hear. So neither of your prompts mentions the `delegate` tool, the other model, or how the backend's messages reach the user. ## How a Conversation Works When the user asks for something that needs the backend, the frontend calls `delegate`, which returns at once, and the frontend acknowledges: "Sure, I'll get on that now." The backend starts working. The user can keep talking. The frontend handles small talk and questions it can answer itself. When the user asks for more work, corrects the work in progress, or says "never mind", the frontend delegates that too, and the backend works out what it means for its work and cancels any tool calls whose results are no longer wanted. When the backend has something the user really needs to hear, such as a result or a question it can't go on without, the frontend relays it in its own words. Everything else the backend outputs (progress notes, function calls, reasoning) reaches the frontend silently, so when the user asks how it's going, the frontend can answer without delegating again. To the user there is one assistant: the frontend never mentions the backend, and talks about the work as its own. ## Speech-to-Speech Frontends The frontend can be a speech-to-speech service, such as [`OpenAIRealtimeLLMService`](https://docs.pipecat.ai/api-reference/server/services/s2s/openai) or [`GeminiLiveLLMService`](https://docs.pipecat.ai/api-reference/server/services/s2s/gemini-live), in place of the STT, LLM and TTS services. Its model then words each request to the backend itself. Two can't be a frontend: `InworldRealtimeLLMService`, because its model calls `delegate` again in place of taking in the backend's messages, and `UltravoxRealtimeLLMService`, because built-in tools such as `delegate` don't reach its session. [`OpenAILiveLLMService`](https://docs.pipecat.ai/api-reference/server/services/s2s/openai-live) has the same shape built in: OpenAI's GPT-Live model is the frontend, and in client delegation its backend is a `BackendLLMWorker`. ## Predictive Delegation > [!WARNING] > > `PredictiveDelegation` is in alpha: its API is likely to change between > releases. A text frontend spends a whole inference deciding to call `delegate` before the backend can start. **Predictive delegation** starts the backend sooner on turns that need it. How it works: on each user turn, a fast yes/no [classifier](https://docs.pipecat.ai/pipecat/learn/classifiers.md) races the frontend's inference, asking whether the turn needs the backend. A "yes" that arrives before the frontend's first output delegates the turn at once and cancels the inference. A "no", a late answer, or a classifier error leaves the turn to the frontend as usual. The frontend is never slowed down waiting for the classifier. ```python theme={null} from pipecat.classifiers.typesafe.jev.classifier import JevClassifier from pipecat.pipeline.llm_with_backend import PredictiveDelegation llm = LLMWithBackend( frontend=frontend, backend=backend, predictive_delegation=PredictiveDelegation( JevClassifier(api_key=os.getenv("TYPESAFE_API_KEY")) ), ) ``` The cost: a classification on every user turn, and a cancelled frontend request each time it delegates early. Predictive delegation needs a text frontend and a request strategy whose `delegate` call takes no arguments, such as the default `TranscriptBackendRequestStrategy`. ## Showing Backend Work in a Client The function calls the backend makes reach clients as RTVI function-call events, the same as calls the frontend makes, so a UI can show what the backend is doing. They are reported at the level [`function_call_report_level`](https://docs.pipecat.ai/api-reference/server/rtvi/rtvi-observer#configuration) sets for them, as any function call is; raise it to show their names, arguments or results. To keep the frontend's `delegate` calls from reaching the client, mark them `DISABLED` on your pipeline worker: ```python theme={null} from pipecat.processors.frameworks.rtvi import RTVIFunctionCallReportLevel, RTVIObserverParams worker = PipelineWorker( pipeline, rtvi_observer_params=RTVIObserverParams( function_call_report_level={ # The frontend's delegate calls aren't reported; every other call is # reported at the usual level "delegate": RTVIFunctionCallReportLevel.DISABLED, }, ), ) ``` While developing, turn on `client_trace` to send every exchange with the backend to the client: ```python theme={null} from pipecat.pipeline.llm_with_backend import BackendConnector llm = LLMWithBackend( frontend=frontend, backend=backend, connector=BackendConnector(client_trace=True), ) ``` Each exchange arrives as an RTVI `server-message` whose `data.type` is `"llm-with-backend"`, which a client's event log shows. ## Running the Backend in Another Process The frontend reaches the backend by name over the bus, so the backend can run in another process. Name the backend worker and run it under its own `WorkerRunner` on a network bus, set up as in [Distributed Agents](https://docs.pipecat.ai/pipecat/learn/distributed-agents.md): ```python theme={null} backend = BackendLLMWorker(name="engineering-backend", llm=backend_llm, context=backend_context) runner = WorkerRunner(bus=bus) await runner.add_workers(backend) await runner.run() ``` In the frontend's process, on the same bus, pass the name in place of the worker: ```python theme={null} llm = LLMWithBackend(frontend=frontend_llm, backend="engineering-backend") ``` The frontend waits for the backend to be ready, so a backend that starts a little after the frontend is fine. ## Adjusting the Defaults `BackendConnector(respond_on_delegate=False)` makes the frontend delegate without replying, for an app where a quick reply on delegating isn't wanted: ```python theme={null} from pipecat.pipeline.llm_with_backend import BackendConnector llm = LLMWithBackend( frontend=frontend, backend=backend, connector=BackendConnector(respond_on_delegate=False), ) ``` The pairing instructions and the way requests are worded are tuned to work across providers. The [reference](https://docs.pipecat.ai/api-reference/server/composite-services/llm-with-backend.md) lists the parameters for custom overrides, and a custom override has to cover everything the default does. Say what is specific to your bot in your own prompts instead. ## Other Multi-Agent Setups `LLMWithBackend` sets up one multi-agent shape: a frontend, a backend, and the messages between them handled for you. When your bot needs a different shape, build it from the multi-agent pieces directly: * [Multiple LLM Agents](https://docs.pipecat.ai/pipecat/learn/multiple-llm-agents.md): agents that each own an LLM, tools and context * [Agent Handoff](https://docs.pipecat.ai/pipecat/learn/agent-handoff.md): moving the conversation from one agent to another * [Job Coordination](https://docs.pipecat.ai/pipecat/learn/job-coordination.md): dispatching work to agents and collecting results * [Distributed Agents](https://docs.pipecat.ai/pipecat/learn/distributed-agents.md): agents across processes and machines ## What's Next For every parameter of `LLMWithBackend`, its connector and predictive delegation, see the reference: > **[LLMWithBackend Reference](https://docs.pipecat.ai/api-reference/server/composite-services/llm-with-backend.md)** > > Configuration, request strategies, predictive delegation, and frames > **[What's Next](https://docs.pipecat.ai/pipecat/learn/whats-next.md)** > > Explore Fundamentals and advanced patterns This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.