billion-context: fold long conversations into restorable summaries so a 100K window runs month-long sessions

Long coding sessions blow up context: quality degrades past the window, or the session dies outright. billion-context (5,244 stars, TypeScript, created August 6, 2026), open-sourced by ranxianglei, sits between any agent and its model API and rewrites Anthropic/OpenAI streams through acp-kernel compression, folding consumed conversation into layered summaries. The distinction is that it is not truncation: the model decides when and what to compress, summaries can be decompressed on demand, and the cache prefix stays intact.

Core features

• Model-driven compression: the proxy injects four tools — compress, decompress, search_context and acp_status — and the model calls them itself as context grows. Compression executes server-side, so folded ranges stay summarized in history until restored.
• Incremental, reversible, prefix-cache friendly: summaries are written in small ranges and decompressed on demand; the project reports a 95–97% prefix-cache hit rate in healthy sessions, with compression itself costing ≤2%.
• Two dozen clients: pi, omp, opencode, dsh, kimi, hermes and zcode get native plugin modes; everything else runs through the bili <client> launcher or a /bili/ baseURL prefix — Claude Code, Codex, Gemini CLI, Qwen Code, Copilot CLI, aider, Crush, Zed, goose and more.
• Paper and benchmarks: the author’s production run covers 4.5 months, three hosts, 174,327 model calls and 18.76B cumulative input tokens with zero window violations on 204,800-token models; a 1,500-item third-party benchmark across LongMemEval-S, ContextBench and MemoryAgentBench places it first on ContextBench at 58.3% lower cost than full context.

Typical use cases

• Month-long sessions: keep one coding session alive for days and decompress the exact range you need when an old detail matters, instead of starting over.
• Long tasks on small windows: move work that used to need a large-context model onto a cheaper 100K-window model, cutting the token bill directly.
• Teams that already run retrieval: summaries plus keyword search via search_context can replace or complement vector RAG, removing an index to maintain.

Quick start

npm install -g billion-context --prefix=~/.local installs the bili command. Three usage modes, in ascending order of intrusion: a native plugin (bili plugin install <client>, proxy lives inside the client and startup is unchanged), a launcher (bili claude, bili codex, no real config file touched), or a baseURL prefix pointing at /bili/. On ports, a manual bili start owns 8787 while each spawned lane manages its own zone from 18787 and remembers its drift, so zero-config installs do not fight existing services for a port.

Relation to existing options

It does not replace a host’s built-in compaction: the built-in summarizer truncates at a threshold, while billion-context keeps reversible layered summaries plus search, at the price of one local proxy hop. Nor is it the same layer as RAG — RAG retrieves an external corpus, this compresses the session’s own history. Like worktrunk covered here earlier, it targets the same engineering friction around parallel agents and long-running tasks: one manages working directories, the other manages context.

Summary

billion-context fits teams with long sessions, small windows and token-cost pressure — especially those already running Claude Code, Codex or opencode. Facts and limits: the license is NOASSERTION, so read the LICENSE before commercial use; the benchmark numbers ship from the author’s own repository rather than an independent lab; MITM modes install a self-signed certificate, so environments with strict traffic-compliance rules should evaluate “requests pass through a local proxy” before adopting; and 5,244 stars arrived fast on a repo created in August 2026, so long-term maintenance cadence is still unproven.