Pipecat: a Python framework that turns real-time voice agents into plug-and-play pipelines

Building a real-time voice agent for support calls, outbound dialing or a voice assistant, the hard part is never the model. It is gluing ASR, LLM, TTS, barge-in detection and the transport layer into a pipeline with millisecond latency — and rewriting that glue every time you switch vendors. Pipecat is an open-source Python framework maintained by Daily (BSD-2-Clause) that turns speech recognition, language models, speech synthesis and multimodal processing into swappable standard components, so you write only the orchestration. It has grown past single-bot territory too: every pipeline is an agent, and agents can hand off to specialists, fan out in parallel, or coordinate over a shared bus — in one local process or distributed across machines. The repo dates to December 2023, sits at 16,847 stars, shipped v1.14.0 on October 24, 2026, and its main branch was still taking merges the day we checked.

Core features

• Pluggable services: every stage — speech recognition, TTS, LLM — ships with ready-made implementations from many vendors (the README’s service table covers OpenAI, Google, Cartesia, Deepgram, ElevenLabs and more). Switching providers is a one-line constructor change; the pipeline structure stays put.
• Multi-agent orchestration: specialist handoff, parallel fan-out, sidecar workers and a shared bus are built in, so an intake agent, a policy Q&A agent and a complaints agent can each run their own pipeline inside one conversation.
• Client SDKs on seven platforms: JavaScript, React, React Native, Swift, Kotlin and C++ are officially maintained, plus an ESP32 port for hardware. WebRTC and WebSocket transports both work, so web, mobile and embedded clients hit the same backend.
• Structured conversations: Pipecat Flows is built in for predefined or dynamic conversation paths with state management — form filling and qualification flows without a hand-rolled state machine.
• A working toolchain: the CLI ships with the package (pipecat init scaffolds a runnable bot in about a minute), Whisker is a real-time pipeline debugger, Pipecat UI offers shadcn-style voice interface components, and Claude Code skills handle scaffolding and cloud deploys.

Typical use cases

• Support and outbound voice bots: connect the phone audio stream, run ASR into an LLM into TTS; interruption, silence detection and latency tracking are framework features, not add-ons.
• Multimodal interactive apps: voice-plus-video coaching, interview practice and interactive storytelling, with React or Swift clients on the official SDKs.
• Local-first privacy setups: every model endpoint is replaceable, so running open-source ASR and TTS locally keeps the whole pipeline inside your network — a common pattern in the r/LocalLLaMA community.

Quick start

Install uv first, then:

uv tool install "pipecat-ai[cli]"
pipecat init

Pick a phone or web bot and your vendors interactively; once the scaffold is generated, follow the quickstart guide to add API keys and you can talk to your first bot. Existing projects can uv add pipecat-ai directly — only the core framework installs by default, with per-vendor extras added as needed.

Summary

Pipecat fits teams pushing a voice agent from demo to production: the framework standardizes transport, interruption and latency plumbing, so swapping models or vendors leaves the pipeline intact, and the client ecosystem is complete. Three honest caveats: it is an orchestration framework, not an end-to-end product — AI service keys and bills are still yours; production deployments lean toward the company’s Pipecat Cloud, and self-hosting means real ops work; and a single-process Python deployment has throughput ceilings, so very large scale means splitting it across processes yourself. The perennial comparison is LiveKit Agents: LiveKit is stronger on the media plane with its own SFU, Pipecat on vendor-neutral orchestration. If your agents need ears rather than a voice, earshot wires ambient audio into coding agents and AudioScript captions system audio in real time — Pipecat is the one that engineers the whole speaking side.