> ## Documentation Index > Fetch the complete documentation index at: https://docs.pipecat.ai/llms.txt > Use this file to discover all available pages before exploring further. # LLMWithBackend > LLMWithBackend pairs a conversational frontend LLM with a backend LLM that handles work needing tools or careful reasoning. > [!WARNING] > > `LLMWithBackend` is in alpha: its API is likely to change between releases. ## Overview `LLMWithBackend` pairs a **frontend** LLM, which keeps the conversation with the user fluent, with a **backend** LLM that handles work needing tools or careful reasoning. It goes where the LLM goes in your pipeline and runs the backend as an agent of its own. The frontend hands work to the backend with a `delegate` tool and keeps talking with the user, and what the backend has to say comes back into the frontend's conversation for it to relay to the user. Composite services hold one or more services and stand in a pipeline where a single service would. `LLMWithBackend` holds the frontend service; the backend is a [`BackendLLMWorker`](https://docs.pipecat.ai/api-reference/server/workers/backend-llm-worker). For a walkthrough, see [LLM with a Backend](https://docs.pipecat.ai/pipecat/learn/llm-with-backend.md). > **[Example](https://github.com/pipecat-ai/pipecat/tree/main/examples/multi-worker/llm-with-backend)** > > An engineering assistant with text and speech-to-speech frontends > **[BackendLLMWorker](https://docs.pipecat.ai/api-reference/server/workers/backend-llm-worker.md)** > > The worker that runs the backend LLM ```python theme={null} from pipecat.pipeline.llm_with_backend import ( BackendConnector, ExplicitBackendRequestStrategy, LLMWithBackend, PredictiveDelegation, TranscriptBackendRequestStrategy, ) ``` ## LLMWithBackend `LLMWithBackend` is a `Pipeline`, not an `LLMService`: functions, settings updates, event handlers and a `FlowManager` go on its [`frontend`](#properties). The backend's tools go in the backend's own context. ### Configuration **frontend** `LLMService` *required* The conversational LLM service, text or speech-to-speech. **backend** `BackendLLMWorker | str` *required* The backend worker, or the name of one registered elsewhere: in the app, or in another process on a shared bus. A worker given here is added to the pipeline worker when the pipeline starts. **connector** `BackendConnector | None` (default: None) How delegation works. A default [`BackendConnector`](#backendconnector) when `None`. **predictive\_delegation** `PredictiveDelegation | None` (default: None) A [`PredictiveDelegation`](#predictivedelegation), to start the backend sooner on turns that need it. Text frontends only. Raises `ValueError` when: * The frontend is `InworldRealtimeLLMService`, `UltravoxRealtimeLLMService` or `OpenAILiveLLMService`, which can't be a frontend (see [Notes](#notes)). * The frontend is speech-to-speech and the request strategy is `TranscriptBackendRequestStrategy`, which reads the frontend's context. * `predictive_delegation` is given with a speech-to-speech frontend, or with `ExplicitBackendRequestStrategy`, whose `delegate` call takes a `request` argument. ### Properties | Property | Type | Description | | - | - | - | | `frontend` | `LLMService` | The frontend LLM service. | | `backend` | `BackendLLMWorker \| str` | The backend worker, or the name of one running elsewhere, such as in another process. | | `connector` | `BackendConnector` | The connector joining the two. | | `predictive_delegation` | `PredictiveDelegation \| None` | The predictive delegation that starts the backend sooner, or `None`. | | `tool` | `FunctionSchema` | The `delegate` tool installed on the frontend. | ## BackendConnector `BackendConnector` manages the interaction between the frontend and the backend. It defines the `delegate` tool, holds the session with the backend for the pipeline's life, and turns what the backend produces into messages in the frontend's conversation. **request\_strategy** `BackendRequestStrategy | None` (default: None) How a `delegate` call becomes the backend's request. When `None`, [`TranscriptBackendRequestStrategy`](#transcriptbackendrequeststrategy) for a text frontend and [`ExplicitBackendRequestStrategy`](#explicitbackendrequeststrategy) for a speech-to-speech one. **frontend\_pairing\_instruction** `str` (default: FRONTEND\_PAIRING\_INSTRUCTION) How the frontend's model works with a backend, appended to its system instruction. A custom override has to cover everything the default does: when and how to delegate, that to the user there is one assistant, what to do after delegating, and what to do with the backend's messages. The request strategy's guidance goes where `PAIRING_INSTRUCTION_STRATEGY_PLACEHOLDER` stands, or after the text when it has none. **respond\_on\_delegate** `bool` (default: True) Whether the frontend replies as soon as a delegation is sent, which is where its acknowledgement comes from. Set `False` to delegate without a reply. **timeout\_secs** `float | None` (default: 30) How long the backend may take to acknowledge a request, including the wait for it to become ready. **client\_trace** `bool` (default: False) Whether to send every exchange with the backend to the client as an RTVI `server-message`, for debugging. See [Frames](#frames). ```python theme={null} connector = BackendConnector(respond_on_delegate=False) ``` ## Request Strategies A request strategy defines the `delegate` tool's parameters and description, and builds the request the backend receives from each `delegate` call. ### TranscriptBackendRequestStrategy The frontend's model calls `delegate` with no arguments, and the request is the transcript of the user and assistant turns since the previous delegation (the whole conversation the first time). The default for a text frontend. Its `delegate` call needs no arguments, so it works with predictive delegation. It reads the frontend's context, so a speech-to-speech frontend refuses it. **frontend\_instruction** `str | None` (default: TRANSCRIPT\_FRONTEND\_INSTRUCTION) The strategy's guidance to the frontend's model, placed in the pairing instruction. **tool\_description** `str` (default: TRANSCRIPT\_TOOL\_DESCRIPTION) How the `delegate` tool is described. **request\_instruction** `str` (default: TRANSCRIPT\_REQUEST\_INSTRUCTION) What the backend should do with the transcript, placed after it. ### ExplicitBackendRequestStrategy The frontend's model words the request itself, in the `delegate` tool's `request` argument. The default for a speech-to-speech frontend, which replies to the user's audio before that audio's transcript reaches its context. **frontend\_instruction** `str | None` (default: EXPLICIT\_FRONTEND\_INSTRUCTION) The strategy's guidance to the frontend's model, placed in the pairing instruction. **tool\_description** `str` (default: EXPLICIT\_TOOL\_DESCRIPTION) How the `delegate` tool is described. **request\_instruction** `str` (default: EXPLICIT\_REQUEST\_INSTRUCTION) What the backend should do with the request, placed after it. ## PredictiveDelegation > [!WARNING] > > `PredictiveDelegation` is in alpha: its API is likely to change between > releases. `PredictiveDelegation` starts the backend sooner on turns that need it. How it works: on each user turn, a yes/no classifier races the frontend's inference, asking whether the turn needs the backend. A "yes" that arrives before the frontend's first output delegates the turn at once and cancels the inference. A "no", a late answer, or a classifier error leaves the turn to the frontend as usual. The frontend is never slowed down waiting for the classifier. The classifier reads what the frontend's model reads: its system instruction, pairing instruction included, the `delegate` tool's description, and the latest messages of the conversation. **classifier** `BaseClassifier` *required* The yes/no [classifier](https://docs.pipecat.ai/api-reference/server/classifiers/overview.md) to ask, such as [`JevClassifier`](https://docs.pipecat.ai/api-reference/server/classifiers/jev). ```python theme={null} from pipecat.classifiers.typesafe.jev.classifier import JevClassifier llm = LLMWithBackend( frontend=frontend, backend=backend, predictive_delegation=PredictiveDelegation( JevClassifier(api_key=os.getenv("TYPESAFE_API_KEY")) ), ) ``` It needs a text frontend and a request strategy whose `delegate` call takes no arguments, such as the default `TranscriptBackendRequestStrategy`. It leaves speculative turns (eager end of turn) to the frontend, and every turn while the frontend filters incomplete user turns. The classifier's metrics are pushed as `MetricsFrame`s. ## Constants The constants are in `pipecat.pipeline.llm_with_backend.prompts`, except the request instructions, which are in `pipecat.workers.llm.backend_llm_worker`, and `DELEGATE_TOOL_NAME`, which is in `pipecat.pipeline.llm_with_backend.connector`. Read the [source](https://github.com/pipecat-ai/pipecat/blob/main/src/pipecat/pipeline/llm_with_backend/prompts.py) for the default texts' wording. | Constant | What it is | | - | - | | `FRONTEND_PAIRING_INSTRUCTION` | The default `frontend_pairing_instruction`. | | `PAIRING_INSTRUCTION_STRATEGY_PLACEHOLDER` | `"{strategy}"`: where a pairing instruction takes the request strategy's guidance. | | `BACKEND_MESSAGE_RELAY_INSTRUCTION` | Appended to each `Backend: ` message for the user: relay it now, once, even if the conversation has moved on. | | `TRANSCRIPT_FRONTEND_INSTRUCTION` | `TranscriptBackendRequestStrategy`'s default `frontend_instruction`. | | `TRANSCRIPT_TOOL_DESCRIPTION` | `TranscriptBackendRequestStrategy`'s default `tool_description`. | | `TRANSCRIPT_REQUEST_INSTRUCTION` | `TranscriptBackendRequestStrategy`'s default `request_instruction`. | | `EXPLICIT_FRONTEND_INSTRUCTION` | `ExplicitBackendRequestStrategy`'s default `frontend_instruction`. | | `EXPLICIT_TOOL_DESCRIPTION` | `ExplicitBackendRequestStrategy`'s default `tool_description`. | | `EXPLICIT_REQUEST_INSTRUCTION` | `ExplicitBackendRequestStrategy`'s default `request_instruction`. | | `DELEGATE_TOOL_NAME` | `"delegate"`. | ## Frames ### Pushed by | Frame | Fields | Effect | | - | - | - | | [`ExternalFunctionCall*Frame`](https://docs.pipecat.ai/api-reference/server/frames/system-frames#externalfunctioncallsstartedframe) | function name, call ID, arguments, result | One per phase of each function call the backend makes. [`RTVIObserver`](https://docs.pipecat.ai/api-reference/server/rtvi/rtvi-observer) reports them to the client as function-call events. | | `RTVIServerMessageFrame` | `data` | With `client_trace=True`, one per exchange with the backend. `data["type"]` is `"llm-with-backend"`, and `data["event"]` is `"request"`, `"output"`, `"tool_call"`, `"error"` or `"idle"`. | ## Notes * **Backend messages in the frontend's context**: What the backend produces is appended to the frontend's context as `developer` messages. A message for the user starts with `Backend: ` and ends with `BACKEND_MESSAGE_RELAY_INSTRUCTION`, and the rest are silent: `Backend (note): ` for text not meant for the user, `Backend (working): ` for a function call the backend is making, and `Backend (thinking): ` for a reasoning summary. * **The `delegate` tool**: The frontend gets `delegate` as a built-in tool: it is sent with every inference beside the frontend's own tools and never enters the context's tool list. * **Services that can't be a frontend**: * `InworldRealtimeLLMService`: its model calls `delegate` again in place of taking in the backend's messages. * `UltravoxRealtimeLLMService`: built-in tools such as `delegate` don't reach its session. * `OpenAILiveLLMService`: it is already a frontend with a backend of its own, set up with its [delegation modes](https://docs.pipecat.ai/api-reference/server/services/s2s/openai-live.md). It can't be a backend either. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.