METR: Anthropic's models are improving about twice as fast as before, and an unreleased 'Model 4' finished training in October

METR, the independent AI evaluator, published a high-access assessment of Anthropic on November 5. Its conclusion: as of October 2026, Anthropic is likely improving its models' capabilities on well-specified tasks at around twice the rate it did between March 2024 and October 2025, and most of that speed-up likely comes from Anthropic's own use of AI to do research.

THE NUMBERS

- Benchmark scores on AECI, Anthropic's version of the Epoch Capabilities Index, have grown 1.9 times faster since October 2025 than before it (95% confidence interval: 1.5 to 2.8 times). METR says that is roughly two years of historical progress in one year.

- METR's guess is that a majority of the current acceleration, perhaps about 80%, is due to AI. It gives that judgment with substantial uncertainty.

- METR says it is confident the speed-up did not come from faster growth in training compute.

THE UNRELEASED MODELS

METR's data includes Anthropic models that are not public, under codenames Anthropic shared with it (METR understands they carry different internal names): Model 2, which finished training in July 2026; Model 3, in August to September; and Model 4, in September to October. Model 4 scores 178.6 on AECI and gets more than 50% on all 15 AECI benchmarks it was tested on, whose names Anthropic kept anonymous. METR says these internal models are the main reason its 1.9x estimate is higher than the 1.7x in Anthropic's Haiku 5.5 system card. In Slack polls, Anthropic staff's average self-reported speed-up from the newest model rose from about 1.2x with Claude 3.7 Sonnet to about 12x with Model 4 in October 2026; METR calls that only very loose evidence.

WHAT ABOUT THE OTHER LABS

METR also charts the frontier across developers, with OpenAI, Google and xAI public models next to Anthropic's. On that view, benchmark scores grew at 1.4 times their pre-October 2025 rate (from about 17 AECI points a year to about 24), not 1.9. METR notes that other developers' internal models are not visible to it, and it lays out both readings: if Anthropic's models were behind the frontier in 2025, 1.4x is closer; if other labs' earlier models were more tuned to benchmarks, 1.9x is.

HOW IT WAS DONE

METR staff evaluated Anthropic from August 12 to October 29, with access to internal metrics on pretraining, reinforcement learning and training compute, benchmark scores for internal models, AI speed-up analyses and interviews with staff. METR says R&D is sped up mainly through AI assistance rather than fully autonomous work, with signs that autonomous contributions will grow soon.

WHAT METR COULD NOT SHOW

Anthropic redacted most of the evidence METR found most informative, saying it was removing confidential information that could help other AI developers. METR says the public therefore has to rely on its judgment more than would be ideal. It saw no strong bottlenecks that would stop further progress in the next 12 months, but says its view of those bottlenecks changed several times and it may be missing some.

WHAT ANTHROPIC SAYS

Sam McCandlish, an Anthropic co-founder who leads much of its safety work, said on X that many of METR's conclusions match his own: AI is speeding up Anthropic's research, though it is hard to say exactly how much. He said Anthropic gave METR complete independence, and that it redacted heavily because the details are commercially sensitive and could give others a roadmap to go faster, a tradeoff he called the right call. Anthropic plans to publish its own analysis of the speed of AI progress inside the company in the coming weeks, including where it stands against its Responsible Scaling Policy.

WHAT WE DO NOT KNOW

How precise the 2x figure is: METR says it would not be surprised if the true acceleration were closer to 1.3x or 3x. And which of the internal models will ship, when, or under what name: METR does not say.