LEANN: Berkeley's open RAG index that shrinks 201GB to 6GB

The expensive part of RAG is not the model — it is the index: every chunk needs a stored high-dimensional vector, and at personal-data scale the index easily runs to hundreds of gigabytes. That is why “fully local RAG” never quite worked. LEANN from a Berkeley team (MIT, 13,774 stars as of November 5, 2026, and the MLSys 2026 best paper, arXiv 2506.08276) flips the design: store no vectors, compute them at query time, and save 97% of the storage — the “201GB to 6GB” number. Zero telemetry, fully local; private RAG on a personal device finally holds up.

Core features

• A store-nothing vector index: trades compute for storage, running one to two orders of magnitude smaller than HNSW-style indexes.
• In-app integrations: works with common LLM clients (Ollama, LM Studio and friends) as a ready RAG application.
• Native MCP integration: mounts as an MCP server so Claude Code and similar agents can query your local knowledge base.
• Zero telemetry: the README states plainly that no data is collected; everything runs locally.
• Cross-platform: Ubuntu/Arch/WSL, macOS (ARM64/Intel) and Windows, Python 3.10–3.14.

Typical use cases

• Personal knowledge-base QA: years of documents, chats and code as a local RAG measured in gigabytes, not hundreds.
• Privacy-sensitive retrieval: internal enterprise documents stay inside the network — the index never lands with a third party.
• RAG on edge devices: storage-limited machines can finally hold the index at all.

Quick start

pip install leann, then follow the docs to build an index and connect an app; the MCP integration lets Claude Code query your local knowledge base directly. The paper (arXiv 2506.08276) has the algorithm and benchmarks. The idea in one sentence: traditional indexes store every chunk’s vector, while LEANN keeps only the graph structure and metadata and recomputes vectors for candidate neighbors at query time — storage drops from O(N) to roughly O(edges), paid for with a batch of model forwards per query. It is not against existing vector databases — once the index fits in a laptop, the default of cloud-first indexing simply loses its necessity.

Summary

LEANN fits developers and researchers moving RAG fully onto personal devices; latency-critical, high-QPS online services should re-price the compute-for-storage trade for their workload. Facts and limits: MIT, Python, an MLSys 2026 best-paper pedigree; query speed depends on local compute, and first-time index builds over large corpora take noticeable time. Together with auto-ontology on this site it completes the “local memory for agents” pair — the ontology layer decides what to trust, LEANN decides what is remembered, and both live on your own disk.