Skip to main content
The development runner gets you a working bot you can talk to on your own machine. Production raises a separate question: what serves the session-start request, decides where the bot process runs, and manages its lifecycle once it’s running?
The development runner (pipecat.runner.run) is not built for production, and is not supported there. It’s a local development tool. Its POST /start endpoint accepts session-start requests from anyone, and some transport webhooks (like Daily’s dial-in webhook) are likewise unauthenticated — so a single request can spawn a bot and create paid resources (rooms) on your account. It has no rate limiting or backpressure, and it’s a single process with no lifecycle management.Running bots in production means choosing one of the options below — not exposing the runner to real traffic. If you’re only prototyping, Exposing the runner for prototyping covers the minimum you’d want in front of it, and where that stops being enough.
There isn’t one right answer for the production version. It depends on traffic shape, isolation requirements, what operational surface your team already runs, and how much of the dispatcher you want to own. This page walks through the answer space at one paragraph per option, links out to concrete patterns, and ends with the concerns that self-hosting puts on you.

​
What plays the runner’s role

The two options below can roughly be summarized as “run everything yourself” and “let a managed runtime do it for you”. Both are presented together here because the decision is the same one (“what serves session-start and runs the bot?”), and most teams evaluating self-hosting are also weighing whether they want to be in that business at all.

​
Option 1 — Build your own dispatcher

You write the HTTP service that receives session-start requests, and you decide how bot processes get into existence. This is the most flexible option and the most work. The two patterns most teams reach for here are:
  • VM per session — your dispatcher calls a cloud provider’s machines API (Fly.io, AWS, GCP) to spin up a fresh VM with the room/token baked into its entrypoint. Strong isolation, easy scale-out, no warm capacity to manage. Pays cold-start latency on every session.
  • Warm pool with subprocess workers — pre-allocate transport resources (e.g. Daily rooms) and a pool of bot subprocesses on one or more long-lived hosts. Replenish on use. Very low session-start latency; constrained by what one host can hold.
These aren’t the only shapes. Some teams build longer-lived orchestrated fleets on Kubernetes (with HPA or KEDA scaling on a custom session-count metric), or use a serverless platform’s function-per-invocation model. The right shape is the one whose latency / cost / isolation profile matches the bot you’re shipping and the traffic you’re seeing.

​
Option 2 — Use a managed agent runtime

This option steps out of self-hosting: a managed platform runs the bot process for you. You supply the bot file (and usually a thin proxy that forwards session-start traffic), and the platform owns the runtime, pool, and lifecycle. This minimizes the operational surface you carry but trades for vendor lock-in and whatever feature ceiling the runtime has. See Managed Agent Runtime. Pipecat Cloud is one example: Daily’s managed runtime, purpose-built for Pipecat. The development runner’s session-start API is deliberately shaped the same way as PCC’s (POST /start, /sessions/{id}/...), which means a bot file and client built against the runner work against PCC unchanged. Other managed runtimes — like AWS Bedrock AgentCore — are the same shape, with their own platform-specific tradeoffs.

​
Exposing the runner for prototyping

Sometimes you need something reachable before you’ve built either option above — a demo, an alpha test, a prototype a handful of people will hit. If you’re going to put the development runner on a public address at all, treat it as prototyping-only. We recommend putting a reverse proxy in front of it that adds, at a minimum:
  • Authentication on POST /start — the runner accepts start requests from anyone. A shared secret or bearer token checked at the proxy keeps strangers from spawning bots (and paid rooms) on your account.
  • Rate limiting — the runner has no backpressure. A per-IP and global request cap at the proxy bounds both load and cost if the address is discovered.
  • TLS termination — the runner speaks plain HTTP.
Note that the runner binds to localhost by default; reaching it from anywhere else (including from outside a container) means starting it with --host 0.0.0.0, which is exactly the exposure this section is about — so keep the reverse proxy, not the runner, as the public entry point. Notably, none of the above addresses the following — which is why this stays prototyping-only:
  • The transport webhooks have a different caller than POST /start. Endpoints like the Daily dial-in webhook are invoked by the transport provider, not your app, so a single proxy secret doesn’t fit them — they’d need provider-appropriate verification (signature checks, IP allowlists) that the runner’s dial-in webhook doesn’t perform. Enabling a telephony transport exposes these endpoints too.
  • None of this addresses lifecycle, capacity, or isolation. You still have a single point of failure, no graceful drain, and no way to scale past one host.
If you find yourself building this out, that’s the signal you’ve outgrown the runner — move to Option 1 or Option 2.

​
Self-hosting considerations

If you build your own dispatcher (Option 1), these concerns are yours to handle. A managed runtime (Option 2) handles most of them for you — though a couple, like bot images and region selection, apply either way. None have a single right answer; the goal here is just to name them so they don’t surprise you.

​
Request authentication

Whatever serves session-start traffic in production needs some form of authentication, both to keep strangers from spinning up bots on your dime and to identify the user the bot is being started for. The shape varies: signed JWTs from your app server, bearer tokens, mutual-TLS, etc. If you’re using a managed runtime this is usually handled by the runtime; if you’re rolling your own dispatcher you’ll need to add it. (This is the same gap that makes the development runner unsafe to expose — see the warning above.)

​
Secret delivery

Bots typically need a handful of API keys (Daily, your TTS/STT/LLM providers, your own service tokens). In development these come from a .env file. In production they should arrive through whatever mechanism your runtime supports — environment variables injected by a secrets manager, mounted files, an external secrets service. Avoid baking keys into images.

​
Bot images and model discipline

In production the bot usually runs as a per-session process, not the always-on development server. Your dispatcher passes the room URL and token into the container’s entrypoint (VM per session) or spawns the bot as a subprocess with those arguments (warm pool); the process runs one conversation and exits. The image’s job is to be a ready-to-run bot — bot file, dependencies, and any cached model weights — not to serve session-start itself. Pipeline models (Silero VAD, any local TTS/STT/turn-detection models) want to be cached at build time, not download time, so the first session doesn’t pay a download. A minimal bot image bakes them in:
The launch command is deliberately left off — it depends on the pattern, since the dispatcher supplies the room and token when it starts the container or subprocess. For anything substantial in local model weights, decide between baking them into the image (faster cold start, larger image) and mounting them from a network volume (smaller image, slower first-touch). Build dispatcher and bot images separately if the two have meaningfully different dependencies. Pipecat Cloud’s base image (dailyco/pipecat-base) is what most of the examples build from; it runs your bot as app.py and provides a sensible runtime environment. It’s a reasonable starting point for self-hosted bot images too, and you can use it without using PCC.

​
Session lifecycle

This is arguably the most distinctive operational fact about hosting bots and worth dwelling on. Most production HTTP workloads are short-lived: a request comes in, you do some work, you send a response, the worker is free again. Bot sessions are the opposite. A single session ties up a process (often a whole container or VM) for the entire duration of a conversation — typically minutes, sometimes hours — and that work can’t be paused, migrated, or interrupted without dropping the user mid-sentence. That shape has consequences that ripple through every other operational choice:
  • Autoscaling can’t be reactive. Traditional HPA-style “CPU is high, add more pods” works because new pods can immediately absorb new requests. With long-lived sessions, scale-up has to be predictive (anticipate demand, warm capacity ahead of it) and scale-down has to be patient (you can’t terminate a pod that’s mid-call; you have to stop sending it new sessions and wait for the existing ones to finish naturally). KEDA scaling on a custom “active sessions” metric is a closer fit than HPA on CPU, but neither solves the “wait for sessions to drain” problem.
  • Pod and VM shutdowns need long graceful periods. Kubernetes’ default terminationGracePeriodSeconds is 30 seconds. For a typical bot host that might have sessions running for 10+ minutes, you need to either crank that up dramatically or accept that deploys will drop in-flight calls. Pair this with a preStop hook (or equivalent) that stops the host from accepting new sessions so the existing ones can drain.
  • Readiness gating. When do you tell the client “your bot is ready”? Returning a join URL the moment you’ve issued the dispatch is fast but lets the client try to connect before the bot has joined. Waiting until the bot has actually joined the transport is more reliable but adds latency to session start. Most teams pick somewhere in the middle: return optimistically and let the client poll or wait for a “bot joined” event from the transport provider.
  • Crash handling. Bots crash. Decide what the client sees, whether you retry, and whether the user-facing transport (Daily room, Twilio call) is reused or recreated. The right answer depends on how long sessions tend to last and how the user perceives a mid-call failure.
  • Graceful drain on deploy. Concretely, this usually looks like: mark the old version as not-accepting-new-sessions, redirect new traffic to the new version, wait for the old version’s sessions to finish (with a deadline), then terminate. Without this you drop in-flight calls every deploy. With it, you accept that “rolling deploys” can take as long as your longest session.

​
Observability

Bots are ephemeral and often distributed across hosts. Correlating events across “the dispatcher that accepted the request, the bot that ran the session, the transport provider that carried the audio” requires designing for it. A single session ID propagated through every log line (and ideally through your transport provider’s session identifier as well) is the load-bearing piece. Beyond that, the metrics most teams find useful for voice bots are active-session count, dispatcher latency to “client ready”, session duration, and pipeline-level metrics from the bot itself (Pipecat’s PipelineParams(enable_metrics=True, enable_usage_metrics=True) gives you a starting point).

​
Networking and placement

Most production deployments hit one or more of:
  • Region selection for latency — voice is latency-sensitive. Bots placed far from users introduce audible delay even when individual services are fast.
  • Service quotas — your cloud provider’s machines API, your transport provider’s rate limits, and the LLM/TTS providers your pipeline calls all have ceilings you’ll find at scale.
  • Telephony-specific networking — SIP and PSTN setups can carry their own constraints depending on the carrier. See Telephony in Production.

​
Where to go next

VM per session

Dispatcher calls a cloud machines API to spawn a fresh VM per session.

Warm pool with subprocess workers

Pre-allocated resources and a worker pool on a single host, replenished on use.

Managed agent runtime

Hand the bot lifecycle off to a runtime that owns scaling and dispatch.

Telephony in production

Webhook-driven dispatch, SIP gotchas, and where telephony differs.