SemiAnalysis’s new GDPx test: The best AI agent scores only 12%

SemiAnalysis has launched GDPx, an agentic benchmark covering 11 domains, from chip design and cybersecurity to recursive self-improvement (RSI). On the v1.0 leaderboard, Claude Opus 5.5 ranks first with a score of 12.0.

Eleven domains, 206 tasks, 14 partners

According to the SemiAnalysis write-up by Max Kan, Andrew Megalaa and Dylan Patel, GDPx covers RSI, chip design, cybersecurity, biology, finance, legal, quant trading, hardware design, materials science, alignment and software engineering. The site lists 206 tasks built with 14 partners. Chip design is the only domain with three partners (Aria, Arcverex and Ergodex). Fireworks supplied inference for open-source models and Modal supplied sandboxes.

SemiAnalysis says it picked domains it expects AI to accelerate most over the next one to two years, rather than following GDP share or the O*NET job taxonomy. It contacted more than 30 reinforcement-learning environment companies, and partners had to pass automated quality checks on the first try. Agents get up to 8 hours per task by default, and some tasks are shorter. The headline number is pass@1 across three attempts, with domains weighted equally and partners weighted equally within each domain.

Each model runs in its native harness where one exists: Claude Code for Opus 5.5, Codex for GPT-6 Astra and Antigravity for Gemini 4 Carbon. Other models use OpenCode.

Opus 5.5 leads, but nobody gets close to 50

The leaderboard lists 13 of the 20 models tested. Scores run from 12.0 down to 2.7:

• Claude Opus 5.5: 12.0
• Gemini 4 Carbon: 10.9
• GPT-6 Astra: 9.6
• Muse Wonder Full: 9.0
• Claude Fable 5.1: 8.5
• GPT 6.1 Sol: 8.4
• GLM 5.3: 2.7

SemiAnalysis calls GDPx “a true frontier-level benchmark”. By domain, Opus 5.5 wins quant trading, hardware design and chip design, GPT-6 Astra wins RSI and materials science, and Gemini 4 Carbon is the best biology model. Gemini 4 Carbon is not yet generally available, the authors note. Muse Wonder Full is the strongest cyber model by a meaningful margin once safety classifiers are accounted for.

That is the authors’ description of binary grading in those two domains, so they report partial scores instead. In a legal redline task, Opus 5.5 averaged 44% partial credit and GPT-6 Astra 25%, with the best single run at 57%. In a finance task with 11 checks, no model solved it; GPT 6.1 Sol averaged 5.6 checks passed and the best single run reached 8.

Safety classifiers drag down two top models

The most unusual caveat is that the overall scores of Opus 5.5 and GPT-6 Astra are described as sandbagged. According to SemiAnalysis, their safety classifiers sometimes block cyber, biology and alignment tasks, which lowers their totals. The post also publishes a ranking that excludes those three domains. In it, OpenAI and Anthropic generally lead, while SpaceXAI, Meta and open-weight models trail meaningfully, and Gemini 4 Carbon is the only model in the same ballpark as Opus and Astra.

So the headline 12.0 mixes two things: raw ability and how often a lab’s guardrails refuse the work. For the cyber domain, partner Arithmetic used model versions that do not trigger the guardrails so results could be shared publicly. In one time-to-exploit test, GPT-5.5 needed more than three hours, while GPT-5.6 Luna and Sol got there in 35 to 50 minutes.

Why the authors say GDPval falls short

The post positions GDPx against OpenAI’s GDPval. SemiAnalysis argues that GDPval tasks are non-interactive, with all context in the prompt and agents limited to web search, a code interpreter and document libraries. It cites 71% human inter-rater agreement on the gold subset and reports broken rubrics, incorrect reference solutions and outdated tasks. It also says its own checks flag nearly 50% of Terminal Bench 4.0 tasks as broken.

GDPx instead uses long-running, interactive tasks from specialist partners. In finance, for example, press coverage describes a deal room of about 2,000 files from a real private equity transaction. That depth is what pushes scores so low, and it leaves headroom for future model releases to show up on the chart.

What the results do and do not establish

Everything here comes from SemiAnalysis and its partners; no independent lab has replicated the results. The authors flag limits themselves: some domains rest on small samples of 20 or 25 attempts, task selection involves editorial judgment, and chip design is, in their words, slightly overrepresented. The post gives no dollar costs per model, though it shows a cost-versus-score chart in which GPT-6 Luna and DeepSeek sit at the low end, Opus at the high end and OpenAI in the middle. Task counts are published only for RSI, finance and biology.

The leaderboard and per-domain pages are public at gdpx.semianalysis.com. SemiAnalysis calls GDPx a living benchmark and invites data companies and labs to get in touch at [email protected].