METR: Anthropic’s models now improve about twice as fast, mostly thanks to AI

The nonprofit METR estimates that Anthropic is improving its models about twice as fast as it did between March 2024 and October 2025, and that AI use inside the company drives most of the gain. Reuters reported the finding on November 5.

A 1.9x jump on Anthropic’s own index

METR published an independent assessment based on a high-access evaluation run from August 12 to October 29, 2026, by about 2.5 full-time staff. Its overall judgment is that Anthropic is improving capabilities on well-specified tasks at roughly 2x the earlier rate, and it would not be surprised by 1.3x or 3x.

On the AECI, Anthropic’s version of the Epoch Capabilities Index, METR puts the acceleration since October 2025 at 1.9x (95% interval 1.5 to 2.8x). The trend rose from 15 to 29 points a year. Different analysis choices gave results between 1.1x and 2.9x.

METR’s best guess is that about 80% of the acceleration seen in October 2026 is due to AI, with substantial uncertainty. It says it is confident the speedup did not come from faster growth in training compute.

Redactions leave readers trusting METR’s judgment

Anthropic held unilateral redaction rights over the report. METR says the company redacted most of the evidence it found most informative, including intermediate measures, estimates of algorithmic progress and forecasts of bottlenecks. Anthropic described the redactions as aimed at removing confidential information that could accelerate other AI developers. METR acknowledges that the public has to rely heavily on its judgment as a result.

It also reports that Anthropic canceled, midway through, the part of the work that would have verified how compute is split between capabilities and safety. METR found no convincing bottleneck in algorithms, compute or data over the next 12 months, but says its confidence is low because its understanding shifted several times.

How Anthropic compares with other labs

For non-Anthropic frontier models, METR’s estimate is a smaller 1.4x, against 1.9x for Anthropic on the same index. That gap is the only cross-lab comparison in the assessment, and METR did not see the internal data of other developers.

Measurement itself is getting harder: METR notes that Anthropic’s internal Model 4 scores above 50% on all 15 benchmarks used. Informal Slack polls of Anthropic staff reached about 12x self-reported uplift for that model, which METR calls “very loose evidence.” In interviews, none of 11 classified researchers leaned against a 2x acceleration and two said no outright.

What the estimate cannot show

METR lists limits in both directions. The figure may be too low if the redacted evidence shows further recent acceleration. It may be too high if non-AI factors explain much of the gain, if gains are confined to auto-gradable tasks, or if Anthropic’s historical growth was unusually low. METR also says Claude was heavily used in its own analysis, which it checked for bias without finding any, and that one team member, who had earlier held equity in a competitor and discussed a role with Anthropic, declined the offer, with their work checked by a colleague.

The assessment is one outside team’s best estimate, built on data Anthropic supplied and partly withheld. Anthropic’s claims about its own research speed, and METR’s, are not independently verified beyond this report. For context on other recent Anthropic safety work, see our piece on Claude’s text watermark.