Goodfire's Nick Cammarata argues neural networks are 'already clean': polysemanticity was mostly tools and training, not superposition
Cammarata's October 15 essay: the polysemantic-neuron evidence was misread; ~99% of InceptionV1's last layer is interpretable. A peer raises five objections.
On October 15, Nick Cammarata — one of the field’s central interpretability authors, publishing at Goodfire — released “Neural Networks Are Already Clean,” a 65-minute interactive essay he labels a working hypothesis. Its central claim: the canonical evidence for polysemantic neurons (single neurons that fire on unrelated concepts) was mostly a misreading of tools and training setups, not proof that networks pack concepts arbitrarily. Cammarata estimates that roughly 99% of neurons in the last layer of InceptionV1 — the field’s early model organism — are understandable under modern architectures, and a preliminary audit of Qwen3-32B suggests about 90% of neuron poles have one main job. If that holds, mechanistic interpretability is in far better shape than the field feared. Within a week, a fellow researcher published five concrete technical objections.
Key points
- Cammarata argues superposition — the standard explanation that networks fit more features than dimensions by packing them together — is the wrong account of InceptionV1’s polysemanticity. The real causes, he writes, were architectural problems and weak feature-visualization tooling, both largely fixable.
- InceptionV1’s late layers were once almost entirely uninterpretable. Retrained with “more reasonable modern regularization,” the same architecture reaches about 70% interpretable neurons; a same-size ConvNeXt reaches about 99%.
- A preliminary audit of Qwen3-32B estimates roughly 90% of neuron poles have one main job, about 7% have two related jobs, and only 2–4% remain unexplained — with a footnote that the number is sensitive to thresholds and not yet independently replicated.
- Researcher Liv Gorton published five technical objections on October 20, arguing the essay’s new reading of the famous 4e:55 neuron as a “V detector” fails the checks she ran.
Background
The field of mechanistic interpretability grew out of a string of polysemantic-neuron discoveries. Anthropic’s “Toy Models of Superposition” and the Circuits series made superposition the mainstream explanation: a network wants to represent more features than it has neurons, so it squeezes several concepts into each one — which is why a single neuron might respond to both cat faces and car fronts. The picture carried a pessimistic edge. If the network’s interior is an arbitrary tangle, understanding it is extraordinarily hard, and alignment work becomes more urgent. Cammarata spent years as a lead author on the Distill and Anthropic work that built this narrative, so publishing a formal retreat — at Goodfire, on his own Substack-grade research page — is why the essay traveled quickly through the field.
The facts
The essay’s checkable claims, all from the author’s side:
| Claim | Number | Source |
|---|---|---|
| Interpretable share of InceptionV1’s last layer (original training) | “almost none” | Essay, section 1 |
| Same architecture, retrained with modern regularization | about 70% | Essay, section 1 |
| Same-size ConvNeXt | about 99% | Essay, section 1 |
| Qwen3-32B neuron poles with one main job | about 90%, plus ~7% with two related jobs, ~2–4% unexplained | Preliminary audit, threshold-sensitive |
| Scope | “models up to trillions of parameters” (Kimi K3 named; details deferred to a companion paper) | Essay, section 4 |
The caveats matter. The whole essay is labeled a working hypothesis. The 90% figure comes from a preliminary audit under “our most generous measure,” with an author footnote that threshold choices move the number. The companion Qwen3-32B audit had not been independently replicated as of this writing. The essay itself is an interactive page with explorable feature visualizations, plus a text-only version; Curt Tigges contributed language-neuron work and Tarun Gogineni edited.
What others say
Liv Gorton, an independent interpretability researcher and author of “Missing Curve Detectors,” published “Some Initial Thoughts on Nick Cammarata’s ‘Neural Networks Are Clean’” on October 20. Her five objections: first, the essay’s reading of 4e:55 — the most famous polysemantic neuron, now claimed to be a V detector — fails her tests, because top activations are animal faces rather than clearly V-shaped, synthetic V stimuli reach only about 25% of maximum response, rounding the V stimuli doesn’t reduce the response, and ablating upstream V-detectors has no effect beyond random. Second, the definition of superposition the essay refutes (“unrelated things packed together”) is not the one prior papers actually used; under the standard definition, packing related features is unsurprising. Third, she reproduced the essay’s channel-crushing experiment and got 61.78% top-1 accuracy against the essay’s 58.35%, seeing Fourier-mode ring structure consistent with superposition theory. Fourth, she argues her own prior work and Olah’s 2023 writing are misrepresented. Fifth, “recursive generalization” doesn’t explain how networks compute. Her conclusion: don’t update much on the essay until its claims are independently checked. She notes Cammarata read a draft and his clarifications are included inline.
Community traction ran cooler than the in-field debate: the Hacker News submission from October 21 sat at 3 points with no comments, never reaching the front page. GitHub did pick up a few reading-notes repositories treating the essay as study material.
Our take
Take it seriously; don’t rewrite your beliefs yet. The essay’s weight comes from its author having built the superposition narrative in the first place, and its interactive evidence — the explorable ConvNeXt late layers, the zoomable Qwen neurons — is new data, not a rhetorical reversal. If “polysemanticity is mostly ordinary engineering problems” holds, interpretability shifts from untangling a knot to fixing a handful of bugs, with consequences for how safety research is resourced. But Gorton’s critique lands where it hurts: the 4e:55 retest is publicly checkable, and if the essay’s new story fails on the field’s most famous example, every other claim deserves a discount. For engineers, the transferable lesson today is method: when a neuron looks unclean, suspect your regularization and your microscope before concluding the network is tangled. What to watch: independent replication of the Qwen3-32B audit, and a direct response to Gorton’s five tests. We have touched interpretability from other angles before — KAN author Liu Ziming’s startup Metacircle, which buys interpretability through architecture, and the interpretability limits of hidden model behavior, where an uninspectable fixed input shapes every inference. This essay adds the layer underneath: whether the neurons are clean to begin with.
What to watch
- Independent replication of the companion Qwen3-32B neuron audit; the essay previews extensions to Kimi K3 and other trillion-parameter models.
- A formal response to Gorton’s five objections, above all the 4e:55 retest data.