Independent review finds Anthropic’s AI R&D pace roughly doubled—but most evidence stays hidden

The story

An independent evaluator given unusually deep access to Anthropic says the company is improving its models on well-specified tasks at roughly twice its previous pace. Model Evaluation and Threat Research, or METR, estimates that benchmark scores for internal systems have grown 1.9 times faster since October 2025 than during the earlier trend, with a 95% confidence interval spanning 1.5 to 2.8 times. The November 5 finding is evidence of acceleration; it is not proof that AI can autonomously build its successor or that progress will keep compounding at the same rate.

METR spent eleven weeks, from August 12 to October 29, examining Anthropic’s AI research and development. The nonprofit says 2.5 full-time-equivalent researchers reviewed internal benchmark scores, pretraining and reinforcement-learning metrics, compute data, staff surveys, code and pull-request activity, AI-agent spending, documents on autonomous contributions and about 15 interviews. The team worked on Anthropic devices, received free model access but no payment, and did not directly sample unreleased models. That access exceeded a normal external evaluation, yet remained mediated by the company.

The headline number comes from AECI, Anthropic’s version of the Epoch Capabilities Index, applied to public and internal models. METR used piecewise linear fits to compare growth before and after a detected breakpoint. Its preferred estimate implies that Anthropic made approximately two years of its earlier benchmark progress in one year. Reasonable analytical alternatives produced estimates from 1.1 to 2.9 times, and METR’s broader subjective range for the current pace runs from 1.3 to 3 times the historical rate. That spread is central to the result, not a footnote.

Attribution is even harder than measurement. METR says it is confident that faster growth in training-compute scale did not cause the acceleration. After considering hiring, data, algorithms and other changes, the team guesses that a majority—perhaps about 80%—of the current speedup is due to Anthropic’s use of AI in R&D. It also says non-AI factors likely mattered substantially in late 2025. Employee interviews and surveys were broadly consistent with a doubling, but self-reported productivity estimates reached an implausibly precise-looking 12-fold in some polls, which METR treats only as loose evidence.

The report therefore does not establish recursive self-improvement. Researchers told METR that AI assistance is the main mechanism: systems write code, run experiments and support human work. They are not yet making highly significant, fully autonomous contributions to the most important stages of model training. METR did hear of early ‘signs of life’ in autonomous work, but details were withheld. A faster human-and-AI laboratory is consequential without being a self-directed machine research program.

The most serious limitation is disclosure. Anthropic had unilateral redaction rights over non-public information and removed a majority of the evidence METR considered most informative. Publicly absent material includes detailed acceleration estimates across the training pipeline, breakdowns of progress from compute, data and algorithms, measurements of key R&D inputs, forecasts for future models and analysis of specific bottlenecks. METR says the public must rely on its judgment more than would be ideal. Anthropic told the evaluator that the redactions protect commercially sensitive information that could help rivals accelerate.

A separate METR review also challenges Anthropic’s August risk report. Anthropic had publicly said AI was significantly speeding its work but probably by less than a factor of two, and it assessed direct catastrophic risk from automated R&D as low. METR agrees that July 15 models posed low direct risk and could not fully automate key R&D. But it says Anthropic’s public wording understated the observed acceleration and that crossing the company’s early-warning threshold was plausible, even though both sides leaned toward concluding it had not been crossed.

Independent reports from Reuters and the Associated Press confirm the assessment’s 11-week scope, the 1.9-fold benchmark estimate and its caveats. AP also reported Anthropic co-founder and responsible-scaling officer Sam McCandlish saying many conclusions matched his view while defending the redactions. That response matters: the dispute is not mainly over whether AI is helping AI research, but over how much, whether the effect generalizes beyond cleanly graded tasks, and how much evidence outsiders should be allowed to inspect.

INNOVOX analysis: the strongest conclusion is procedural. Frontier laboratories can now measure AI-assisted research acceleration using internal models and operational data, and an outside group can interrogate those measurements. But a company-controlled audit with decisive evidence hidden cannot provide the same confidence as a reproducible scientific result. The report should move governance away from anecdotes about coding productivity and toward repeatable indicators, access rules and disclosure standards that can survive commercial pressure.

What to watch next is whether the apparent speedup survives broader tasks and other laboratories. METR found no convincing near-term bottleneck in algorithms, training compute, experiment compute or data, but repeatedly changed its view as new information arrived and says it may have missed constraints. Policymakers and safety teams should not extrapolate a neat exponential curve from one company’s internal index. They should demand faster evaluation cycles, independent replication and clear triggers for slowing deployment if AI moves from assisting researchers to autonomously changing the training pipeline.

INNOVOX analysis

The report is stronger than a productivity survey because it connects internal-model benchmarks, training inputs, interviews and unreleased systems. It is weaker than a reproducible audit because the developer controlled disclosure of the decisive evidence. The central policy result is therefore a credible warning signal, not a settled measurement of recursive self-improvement.

What to watch

Watch for an unredacted or reproducible benchmark series, equivalent assessments at other frontier laboratories, evidence on messy real-world research tasks, and whether faster model progress forces shorter safety-evaluation cycles. The most important near-term test is whether AI begins making significant autonomous contributions to core training decisions rather than primarily assisting researchers.