METR audit of Anthropic internals: AI R&D runs 1.9x faster, and the July warning threshold may already be crossed
METR's 11-week audit finds Anthropic's AI R&D running 1.9x faster, ~80% of it attributed to AI itself — and its July warning threshold may already be crossed.
METR (Model Evaluation and Threat Research), the nonprofit that independently evaluates frontier models, published on November 5, 2026 the kind of audit no AI lab has allowed before. From August 12 to October 29, two and a half full-time METR staff worked through Anthropic’s internal benchmarks, training metrics and staff interviews, and concluded that since October 2025 the capability of Anthropic’s models has been improving about 1.9x faster than before (95% CI 1.5–2.8), with roughly 80% of that speedup attributed to using AI to do AI research itself. Measured against the “2x acceleration” warning line in Anthropic’s own Responsible Scaling Policy (RSP), METR considers it plausible the threshold had already been crossed at the July 15, 2026 assessment date.
Key points
- Capability gains run at 1.9x. On METR’s AECI composite benchmark, Anthropic models gained roughly 15 points a year before October 2025 and about 29 points a year since; the statistical estimate is 1.9x (95% CI 1.5–2.8), with METR’s subjective range reaching 1.3–3x.
- About 80% is attributed to AI itself. METR estimates “perhaps ≈80%” of the current speedup comes from using AI to do research, a share that grew over time; an acceleration of training-compute scaling was explicitly ruled out.
- The warning threshold may be passed. Against the RSP’s 2x line, METR says it is plausible the threshold was crossed as of 2026-07-15. Anthropic’s August risk report said publicly “less than a factor of 2”; the unredacted internal version implies 1.97.
Background
The question METR tested comes from Anthropic’s own commitments. Its Responsible Scaling Policy defines a 2x acceleration warning level: when the rate of progress in aggregate AI capabilities doubles — one year delivering two baseline years of progress, plausibly attributable to research automation — a warning evaluation should follow. The risk report Anthropic published in August settled on “meaningful acceleration … though by less than a factor of 2.” METR’s assessment is an external check of exactly that section: no payment from Anthropic, work confined to Anthropic devices, and access to the internal AECI benchmark, training-run metrics, self-reported productivity surveys and about 15 interviews with researchers. Safety claims at frontier labs are otherwise self-reported, which is why this level of access has no precedent in the industry. The scale behind the acceleration is something we have followed before: Anthropic’s amended IPO filing lifted its compute commitments to roughly $895.4 billion — fuel was never the constraint; the question is what sets the speed.
The facts
The key numbers from METR’s report, published November 5, 2026:
| Metric | Value |
|---|---|
| Capability growth (after Oct 2025 vs. before) | 1.9x, 95% CI 1.5–2.8 |
| METR subjective range | ~2x, spanning 1.3–3x |
| Industry-wide estimate (incl. OpenAI, Google, xAI models) | ~1.4x |
| AECI growth per year | ~15 pts/yr → ~29 pts/yr |
| Speedup attributed to AI use | “perhaps ≈80%” |
| Self-reported productivity gain | 1.2x (Claude 3.7 Sonnet) → ~12x (internal Model 4, Oct 2026) |
| Breakpoint in growth rate | 2025-10-07 |
| Risk report wording: public vs. internal | “less than a factor of 2” vs. 1.97 implied |
The caveats deserve to survive the summary. The industry-wide figure drops to 1.4x for two possible reasons METR offers: Anthropic lagged the frontier before 2025, or its historical trend line understated early progress and the gap closed. Staff self-reports are called “only very loose evidence”; Anthropic’s own Haiku 5.5 system card self-reported just 1.7x, with internal-only models accounting for most of the difference. Methodologically, METR fit piecewise linear regressions with bootstrap confidence intervals, and the report is categorical on one point: “We are confident that the acceleration was not due to an acceleration in the scaling of training compute.” Three unreleased internal models (Model 2/3/4) finished training between July and October 2026; the strongest, Model 4, scored 178.6 on AECI and cleared 50% on all 15 of its benchmarks.
What others say
Anthropic’s position sits inside METR’s companion review: redactions were “aimed at removing confidential information that could accelerate other AI developers,” and Anthropic’s researchers lean toward the threshold not having been crossed — while the company’s own internal report concedes its attribution estimates are uncertain enough that “it seems plausible we do exceed this factor of 2 overall.” The redactions are the sticking point: most of the evidence METR rated most informative (forecasts for future models, per-training-step breakdowns, bottleneck analyses) was stripped, so the public “has to rely on METR’s judgment much more than would be ideal.” A compute-allocation audit — checking how much compute goes to safety research — was canceled by Anthropic mid-engagement; Daniel Kokotajlo, author of AI 2027, argued on X that such audits are what make slowdown commitments verifiable in the first place. Researchers on the ground do not agree with each other either: one told METR “it is hard to see how I am not being 3x more productive,” another credited AI with only 15–20% of the change, and a third pointed to hiring scale as the biggest driver. In internal polls, employees said they are “only bottlenecked on ideas and compute, no longer bottlenecked on code.” AP’s coverage framed the report against the wider push by experts to slow development down: the outside debate says ease off, the internal data says the pace is climbing.
Our take
For developers, the durable finding is not the 1.9x itself — it is that a lab could publish “less than a factor of 2” while its unredacted internal figure implied 1.97, and only a third party with eleven weeks of access could surface the gap. Independent audit is worth having, but an audit whose most load-bearing evidence can be deleted unilaterally deserves the discount METR itself applies. Be honest about the weak points too: AECI covers well-specified benchmark tasks, overfitting is not excluded, and nobody should treat a 12x self-reported figure as data. Even on the most conservative industry-wide reading of 1.4x, though, AI accelerating AI research is no longer a thought experiment — and both METR and Anthropic agree direct catastrophic risk remains low for now; the warning is about speed, not imminent danger. As execution shifts to models, the human-side counterweights need to move at the same pace: OpenAI already runs activation probes that monitor model internals token by token, and a Claude Code engineer has warned about cognitive offloading — faster tools should mean stricter judgment, not just faster judgment.
What to watch
Three things are worth calendar space. First, Anthropic’s itemized response: METR has published the public and private wordings side by side, so whether the next risk report revises the “less than a factor of 2” line is the first signal. Second, whether other labs accept the same treatment — METR has published its methodology and proposal, so the bar to entry is documented; what is missing is willingness. Third, the timeline METR leaves behind: it “cannot rule out a substantial further acceleration” over the next six months, enough to move RSP risk levels off “low” quickly. The date 2026-07-15 is therefore not just a historical footnote; it is the baseline the next assessment will be measured against.