Search#Web scraping

TinySearch: local search and reranking, only useful passages

A self-hosted web research MCP server: no paid search key, local crawling and BM25 plus embedding rerank, returning only source-linked original passages.

Project and installation docs

View project

https://github.com/TinySuiteHQ/TinySearch

When an agent researches, “search, then read whole pages” is where the tokens go: navigation, footers and off-topic paragraphs all get billed. TinySearch moves the filtering in front of the model. It searches and crawls on your machine, reranks with BM25 plus embeddings, and hands over only the relevant original passages with their source URLs. It had about 230 stars as of 2026-10-06.

What it does

  • Lightweight search: search returns titles, URLs, previews and dates without starting a browser or loading an embedding model, and domains hard-limits results to chosen sites.
  • Question-focused reading: scrape_urls reads one to five URLs; with a focused query it chunks and reranks each page and returns only relevant passages, and without one it returns cleaned Markdown in page order within a token budget.
  • A browser only when needed: browser_navigate and browser_act step in when content appears only after interaction.
  • No rewriting: every passage is the page’s original text, not a model summary, so what you cite is what the page said.

Who it’s for

  • Developers whose Claude Code or Cursor sessions look things up constantly and who want research to cost less context.
  • People who’d rather not sign up for a paid search API and want the whole retrieval loop on their own machine.

Setup

Needs uv. Client config:

{
  "mcpServers": {
    "tinysearch": {
      "command": "uvx",
      "args": [
        "--python",
        "3.12",
        "--from",
        "tinysuite-search[server]",
        "tinysearch"
      ]
    }
  }
}

The first scrape initializes Chromium and the embedding model; uvx --from "tinysuite-search[server]" tinysearch setup warms both up ahead of time.

Our take

It fills the gap between paid search APIs and the official Fetch server: keyless search, and page reads that return only filtered passages. The repo ships a reproducible benchmark script; by its method TinySearch uses about 64% fewer tokens than search-then-read-whole-pages, though real savings depend on pages and settings. Note that the default backend is the DDGS library, which scrapes results from DuckDuckGo and other engines. It can be rate-limited and sits in a terms-of-service gray area; for steadier results use the Docker setup with a bundled SearXNG. The project is young and mostly maintained by an individual. Licensed MIT.