METR finds AI task horizons still doubling every 3.6 months, judgment still lagging
METR's November 20 update puts task horizons at a 3.61-month doubling and about 27 hours on verifiable tasks, while judgment tasks still trail human experts.

METR (Model Evaluation and Threat Research), the nonprofit that independently evaluates frontier models, published its fourth-quarter capability update on November 20, 2026. Pooling its older TH1.1 task set with the new TH2, METR finds that time horizon — the length of a task a model can finish on its own at 50% reliability, measured in how long a human expert would need — is still doubling roughly every 3.61 months, with a 95% confidence interval of 2.08 to 4.94 months. The same report lists a gap on “judgment” tasks as one of its main takeaways: where correctness cannot be checked quickly, frontier models still fall short of human experts.
Key points
- The doubling rate has not moved. 3.61 months (95% CI 2.08–4.94), or 110 days, which METR calls similar to the 105-day trend on TH1.1 alone. It is a new estimate on a combined suite, not a revision of the ~7-month figure from METR’s 2025 paper.
- Verifiable work tops out around 27 hours. On the verifiable subset of TH2, current public frontier models show a time horizon “of up to 27 hours”; fit that subset on its own and the doubling time drops to 1.65 months (95% CI 0.79–3.99).
- Judgment is the weak spot. Five of the 36 TH2 tasks are hard-to-verify or judgment tasks, and on all three shapes — rating philosophical arguments, designing an evaluation, grading another model’s task — the models’ judgment came up short against the human reference.
Background
In March 2025, METR proposed measuring capability by how long a task takes a human, and fit a doubling time of roughly seven months at the 50% reliability threshold. That suite settled into TH1.1: 222 tasks drawn from SWAA, HCAST and RE-Bench, the longest around 30 hours. The ruler wore out fast. Within a year models had saturated TH1.1, and by 2026 METR had trouble recruiting human baseliners who could outperform them. So it changed the test: TH2, 36 tasks in 12 families, eight families automatically scored (optimization, machine learning, puzzles, games) and four hard-to-verify or judgment families (conceptual reasoning, judgment, hard-to-verify research). That is why the update does not read like another gain: it is a new ruler. The tasks are harder, so the same model gets a lower time horizon on the combined suite than on TH1.1 alone.
The facts
The numbers from METR’s November 20, 2026 update:
| Metric | Value |
|---|---|
| 50% horizon doubling, TH1.1 and TH2 combined | 3.61 months, 95% CI 2.08–4.94 (110 days) |
| Previous TH1.1 trend | 105 days |
| Doubling time on the TH2 verifiable subset alone | 1.65 months, 95% CI 0.79–3.99 |
| 50% horizon on TH2 verifiable | up to about 27 hours |
| Pooled 50% horizon | GPT-6 Astra ~40 hours, Opus 5.5 ~39 hours (95% CI 13–2645 and 12–1253) |
| Longest score threshold in the suite | 200 hours; naive extrapolation reaches it around July 2027 |
| New tasks | TH2: 36 tasks, 12 families |
| Models tested | 15, from GPT-4o through Opus 5.5 and GPT-6 Astra |
Read the 27-hour figure carefully. METR’s text says “up to 27 hours,” names no model and gives no confidence interval; the same batch of models lands at 39 to 40 hours on the pooled suite. The two are consistent — TH2 tasks are longer and harder, so a harder subset yields a lower horizon for the same model. The warning is in the intervals: Opus 5.5’s 39 hours comes with a range of 12 to 1253 hours, GPT-6 Astra’s 40 hours with 13 to 2645. METR explains the width in an appendix — when there are few task points far out in time, the logistic fit has little information about where the curve should fall, so point estimates stretch out and the intervals stretch with them. The newest Anthropic model in the run list is Opus 5.5; Fable 5.2, released November 9, is not in this round.
The update also introduces a metric: Lifecycle Cost. METR defines it as the estimated cost of building a containerized environment for a task, then running and scoring it 100 times. An LLM judge (Opus 5.5) produces the estimate from the task’s code, README, instructions and scoring rules, calibrated until it roughly matched METR’s own estimates on about 10 tasks across domains. The point is to price the work that makes an eval realistic — manual scoring, GPU time per run, serial time, token spend, action monitoring, environment complexity, interaction with the real world. METR’s conversion: for a task of fixed length, doubling lifecycle cost is roughly equivalent to halving the agent’s time horizon, and in their data a doubling delays a model’s 50% success point by about 4.6 months. Human completion time and lifecycle cost correlate at R² = 0.82 across their tasks.
The judgment gap
METR defines judgment as the ability to make good choices without clear, fast empirical feedback. Its four judgment families hold five tasks, and the results come in three shapes. The first is LMCA, where models rate critiques of philosophical arguments and are scored on agreement with expert ratings: many models now beat the agreement level of experts who spent five minutes on the task, but every public model remains below experts who spent fifteen. The second is Making METR Tasks, where an agent must design an evaluation METR would find informative; “none of the tasks agents produced were sufficiently useful to us to warrant inclusion in this post.” Models gravitated to automatically scored optimization tasks even when told to build something whose success depends on decisions with poor short-term feedback, and despite internet access they did not try to learn what METR wanted. The third is Judge Run, where models grade tasks other models wrote: agreement with METR’s own ratings is high when the author is weaker than the judge, and inflated when the author is as strong or stronger. Opus 5.5, the judge that agreed best with the human scorer, rates GPT-6 Astra’s tasks about 0.2 higher out of 1.
METR supplies two counterweights. On machine-learning evaluations that run for days to weeks with programmatic scoring, models beat the human baseliners METR recruited, and P-Zero Research’s TasteVal reports similar results. And models are probably underelicited here: they visibly leave token budgets and tools unused. So the finding is local — the gap sits in work with slow feedback and no cheap check, not in judgment as a whole. MIT Technology Review adds the direction of travel: Opus 5.5 now rates “borderline above the bar” on the evaluation-design task, where a year ago every model was “clearly below the bar.”
What others say
MIT Technology Review calls this “the most misunderstood graph in AI,” widely shared and widely misread because the measures are complicated. It plots paired task length — how long a human needs, whether the model can finish alone — not general capability, and the original version simply omitted hard-to-verify work, which is both hard to train on and hard to score. The headline number also drifts as it travels: MIT TR describes the original plot as doubling “every four months,” while METR’s 2025 paper says about seven months. METR’s paper says the only range it is fairly confident about is “around 1-4 doublings per year,” and both figures sit inside it. MIT TR also notes that parts of the field are still absorbing METR’s heavily redacted report from earlier in November, which estimated that AI had sped Anthropic’s R&D up by about 1.9x and raised fears about recursive self-improvement; we covered that audit in detail.
Outside comment centers on extrapolation. Sayash Kapoor, a computer scientist and incoming professor at UC Berkeley, welcomes the shift: “We’re at the point where AI evaluations need to get more creative.” Najoung Kim of Boston University says the improvement says something about now, but “I’m not sure just yet how to think about how it would extrapolate.” A METR staff member, Matt Putz, is blunter: hard-to-verify or judgment-laden tasks are “much harder to study, and especially to quantify,” and he does “not think we’ll be able to establish trends as clean and robust as for the easy-to-specify” ones.
One cross-check is possible. METR notes that newly published OpenAI data on internal research tasks implies a doubling time of about 3.8 months, close to its own 3.6. OpenAI’s own wording is more careful: from January to July, success rates generally rose across difficulty buckets where a ground-truth outcome exists, and over the last six months more than half of successful 4–8 hour tasks involved at least one human intervention. The pace held and the work still needs a hand — both can be true, because they describe different things.
Our take
For anyone building products, the useful part is not 3.61. It is that the bottleneck moved. Where output can be scored automatically, models already work in tens of hours; another point of benchmark headroom buys little. The gap opens where feedback is slow — picking a direction, cutting a feature, deciding whether an output is trustworthy — and that is exactly where METR measured models behind people rather than where intuition says they should be.
The base is thinner than the numbers suggest. Only three of the 36 tasks have real human baselines; the (score, time) points for the other 31 come from researcher estimates. METR paid $200–350 an hour, roughly $90,000 in total, and models still beat most of the people it hired. The 27-hour figure has no interval, and the pooled interval is too wide to forecast with. Nor is this a plateau or a failure: 3.61 months is faster than seven, and METR’s own language is that capability is still rising quickly. What changed is the ruler and the error bars.
The one thing worth taking from this report is the lifecycle-cost conversion: double the cost of a task and you roughly halve the model’s horizon. It gives evaluators a budget language. Getting closer to real R&D means paying for expert scoring, long serial runs and real-world interaction — costs that never appeared on a benchmark table.
What to watch
Three things. First, five judgment tasks is a small sample; whether METR can fit a trend at all once it widens that pool is the open question, and METR says such trends will not be as clean as for well-specified work. Second, OpenAI’s internal numbers and METR’s 3.6 months corroborate each other, but neither method is public enough to reproduce — whoever publishes first wins the argument. Third, the July 2027 date for the 200-hour threshold deserves repeated testing: it assumes the doubling rate holds, and METR is explicit that it remains uncertain whether high-cost tasks double at the same rate.