Astra's 99.9% lived on OpenAI's own test setup. The independent rerun says 62.7%, and the AGI claim missed its metric

OpenAI's GPT-6 Astra reached subscribers on September 4 with an "AGI era" declaration and a benchmark gap, and both statements arrived from the same briefing.

The headline claim: Astra saturates ARC-AGI-3, a benchmark built to resist memorization, at 99.9 percent.

Independent evaluation contradicts nothing about the underlying run. ARC Prize, the organization behind the benchmark, reran Astra on its provider-neutral Standard test environment on launch day.

ARC Prize gave Astra 62.7 percent on the Standard harness.

A 37-point spread is not noise. It signals that the test environment, custom prompting, and API access built into a provider's own setup can add a third of the headline to the score, and that whoever owns the testing owns the number.

OpenAI president Greg Brockman still closed the Thursday briefing by declaring "Welcome to the AGI era." He hedged in the same breath.

"I do leave it up to the reader to decide for themselves if this qualifies for them," Brockman said, per briefing coverage.

What was missing is the fact to price

OpenAI's charter defines AGI as systems that "outperform humans at most economically valuable work."

GDPval, the benchmark OpenAI built to measure exactly that, did not appear in the launch materials.

Independently, Artificial Analysis ran its own variant and found Astra gained roughly 80 points on long-horizon knowledge work while declining on GDPval categories including banking support and scientific coding, the reporting said.

The composite shows the same split. On the Artificial Analysis Intelligence Index, Astra scored 61.2 against 60.9 for GPT-5.6 Sol, and 65.7 for Claude Fable 5.1 from Monday's Anthropic release.

Extraordinary in terminal math, exploit building, and computer use. Ordinary in some categories of real work.

The monitoring costs 20% and is "fragile"

The chief scientist discussed the governance side in the same briefing.

Chain-of-thought monitoring (reading the model's written reasoning to catch it misbehaving) is "fragile," and "unfortunately trending in a negative direction," Jakub Pachocki said, per the briefing coverage.

That system carries a measurable bill. The 20 percent compute overhead on all monitored inference makes every Astra call costlier than the model alone, the reporting said, and is paid for on every workload at every subscriber, safety-critical or not.

Astra is the first model rated Critical in cybersecurity under the Preparedness Framework, the tier at which a model can develop working zero-day exploits unaided. Daybreak access for defenders runs through Accenture, IBM, CrowdStrike, Cisco, Sophos, and Cloudflare as vetted partners.

"Extra safety checks can sometimes slow, pause, or stop legitimate work, including defensive cybersecurity," OpenAI research VP Amelia Glaese said, per briefing coverage.

No law enforces that governance. The AI Kill Switch Act, filed by Representatives Ted Lieu and Nathaniel Moran, would fine noncompliance up to $20 million per day, and it is not enacted.

How Astra was trained matters here

Astra is the product of OpenAI's largest training run, more than 100,000 GPUs at the Abilene facility.

The new factor: alignment feedback came partly from other AI models acting as supervisors, in place of human evaluators for some of the signal, the briefing said. When judge and judgee are both models, the guarantee behind "stays in scope" gets harder to audit.

Jailbreak refusals hold at 91.5 percent against 59 percent for the previous model, so the guardrails work most of the time.

I get the sense buyers now face a category question rather than a recall question, and I would not call either side dishonest. The claims came with a built-in condition, and most coverage carried the number without the condition.

Source