The AI Sift is part of you-do-nothing

← Back to The Latest

Anthropic warns it may soon reach AI that improves itself

Anthropic said it may soon hit its own threshold for AI that speeds up its own development.

The decision it puts on your desk

Write down by Nov. 20 which steps of your model work AI already does, and what would show you a jump you missed.

Anthropic warned on Nov. 2 that Claude may soon reach AI that improves itself faster than people can check it.

The warning sits in risk factors the company added to a revised filing that day. Anthropic already uses AI to build new models. Its safety policy names the point where that work turns sharply faster.

TechCrunch illustration for Anthropic's recursive self-improvement (RSI) warning (Source: TechCrunch)
TechCrunch illustration for Anthropic's recursive self-improvement (RSI) warning (Source: TechCrunch)
Where Anthropic's current models sit against the bar in its own Responsible Scaling Policy
Where Anthropic's current models sit against the bar in its own Responsible Scaling Policy

The filing language

Anthropic's policy for risky scaling sets a bar it calls dramatic progress driven by AI. Internal estimates for future models suggest the company may soon meet that bar, the filing said. No current model sits above it.

The company also said it has missed such a jump in hindsight before. "We have in the past not identified, and may in the future not identify, a significant acceleration in our rate of development until after it has occurred," the filing said.

The evidence behind the claim

Recursive self-improvement describes AI that gets better at building the systems that build its successors. Researchers have argued about the idea for years. Anthropic now puts it in a document that carries legal weight.

Anthropic gave no benchmark, no date, and no measure of distance to the bar. The company said it keeps assessing the data behind the claim. Critics noted that listing paperwork also works as marketing for a firm heading to public markets.

Capability work and safety work, and what the filing says about the gap between them
Capability work and safety work, and what the filing says about the gap between them

Commitment on human control

Anthropic said it does not expect to run autonomous model builds without people in the loop. "We do not anticipate being able to safely use Claude to fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding, if and when that becomes technically possible," the filing said.

The company is considering updates to its policy to keep people in control of frontier model work. That promise holds only while the bar stays uncrossed. The filing does not say what follows once it is crossed.

Claude's coding, math, and cybersecurity results have grown sharply through 2026, and Anthropic now runs toward a public listing. What the document adds is a date on the calendar. A self-improving system no longer reads as remote.