earshot: give your coding agent ears

AI coding agents can now write the code that renders a film, a product demo or a game, and they still cannot hear what they made. They cannot tell a premium key press from a cheap one, notice a hit landing 80 ms after the cut, or hear that the music swells in the wrong place. earshot is an open-source Python toolkit by Leon Lin (MIT, 2,515 stars as of October 23, 2026) that turns sound into evidence an agent can reason about: what a file sounds like in words, how well it fits a description, the best moment inside a long recording, whether it is a real recording or code-synthesized, a second-by-second listening transcript of a whole film, and a scorecard against professional launch films. It runs locally on CPU and connects as a CLI, a Python API or an MCP server.

Core features

• Hearing and judging: describe returns a calibrated sound description, match scores how well a file fits a description (strong / plausible / weak / no match), and find locates the best moments in a long recording as trims. On a held-out library of 125 real recordings, the project reports a “strong” verdict is the right kind of sound 98% of the time and the exact variant 61% of the time.
• Real versus synthetic: realness separates real recordings from code-synthesized audio at a held-out ROC-AUC of 0.92; about a fifth of clips come back “uncertain”, and the README admits the model can be fooled by good synthesis.
• Film listening and grading: timeline produces an event-level listening transcript of a whole film (times, levels, tags, sync to picture cuts), sheet renders an annotated PNG for image-reading agents, and critique grades a launch video against a profile of 7 professional films with timestamped fixes. A 60-90 s film takes about 1.5 minutes on a laptop CPU.
• earshot forge: the agent writes a sound as code — physical synthesis recipes of modal resonators, filtered noise and a computed room — and forge renders candidates, scores them on description match and distance to real references, and evolves the best. Every sound in the launch film was made this way.
• A bank of 252 sounds: version 0.4 adds 51 sounds styled for music-led launch films (rounded UI pops, wood and plastic knocks, cut hits with a kick, gated glockenspiel, water drops, camera shutters), rendered by id with no models and no network; a launch preset places each effect against the music bed and the cut.
• earshot fit: translates a recording back into code — it fits a parameterized synthesis model and writes a recipe module that re-renders the sound, varies its pitch and damping, or morphs between two recordings.
• MCP server: one claude mcp add wires the same tools into Claude Code, with models staying warm (each cold CLI call spends 20-40 s loading them).

Typical use cases

• Sounding code-made films: tools like video-use handle the editing; earshot adds the listening — check sync after a render, fix what the scorecard flags, then pull effects from the bank.
• Searching an effects library: a few hundred downloaded samples ranked against one sentence, with the best match first.
• Brand-audio QA: build a profile from launch films you admire, then grade every new cut against your own standard.

Quick start

The README asks for Python 3.10+ and ffmpeg on PATH. CPU only; no GPU is used:

git clone https://github.com/Leonxlnx/earshot && cd earshot
python -m venv .venv
.venv/bin/python -m pip install torch --index-url https://download.pytorch.org/whl/cpu   # Windows: .venv/Scripts/python
.venv/bin/python -m pip install -e ".[mcp,llm,dev]"
.venv/bin/python -m earshot doctor --load

First use downloads two open models (about 1.1 GB: CLAP and an AudioSet tagger) into a local cache; after that earshot reads the cache only and can run fully offline. Run earshot describe enter.ogg and you get your first description back in words.

Summary

earshot suits anyone producing video, animation or games through coding agents who cares about the audio half, and content pipelines that want listening as a QA step. It does not suit real-time or latency-sensitive work.

Limits to know up front. CPU-only means slow: every CLI call is a fresh process that loads the models for 20-40 seconds, so batch work belongs in the MCP server or the Python API, and even a 60-90 s film takes about 1.5 minutes for a timeline. The project is days old — the repository was created on October 15, 2026, and 0.4.0 shipped on October 22 — so 2,500 stars in nine days have had little community validation, and the blind tests behind some numbers had one listener and about fifteen trials, which the README itself calls a small test, not a proof. It cannot judge taste or emotion, cannot transcribe speech content, and cannot check licences for you; the forge’s reference recordings are separately licensed and not in the repo, so --refset needs your own. The optional llm critique calls Gemini and requires your key plus explicit upload consent. The repository is not archived and was last pushed on October 22, 2026. We verified the README, CHANGELOG and release page, and we did not install or run it.