New GDPx Test: Best AI Agent Scores Only 12%
SemiAnalysis launched GDPx on Nov. 2, a benchmark of 206 agent tasks in 11 high-value domains. Claude Opus 5.5 led with 12.0, ahead of Gemini 4 Carbon (10.9) and GPT-6 Astra (9.6), while finance and legal tasks left most models near zero.
Research firm SemiAnalysis on Monday, Nov. 2, launched GDPx, a benchmark that drops AI agents into realistic work environments across 11 high-value domains, from chip design to legal drafting. Anthropic's Claude Opus 5.5 topped the first leaderboard, yet it completed only 12% of the work.
Key takeaways
- GDPx v1.0 has 206 tasks in 11 domains, built with 14 specialized data partners, and ranks 13 models.
- Claude Opus 5.5 scored 12.0, ahead of Google's Gemini 4 Carbon at 10.9 and OpenAI's GPT-6 Astra at 9.6.
- SemiAnalysis says safety classifiers pulled down Opus 5.5 and GPT-6 Astra in cyber, biology and alignment tasks.
What the GDPx benchmark tests
The benchmark, written up by Max Kan, Andrew Megalaa and Dylan Patel, covers chip design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement, hardware design, materials science, alignment and software engineering. Government, real estate and insurance were left out on purpose.
Each task runs for up to eight hours by default. Models use their own harness where one exists, such as Claude Code for Opus 5.5 and Codex for GPT-6 Astra, with the open-source OpenCode as the fallback. Scores are pass@1 averaged over three attempts, and every domain carries equal weight.
"GDPx is a true frontier-level benchmark: even the best model scores just 12%," SemiAnalysis wrote in its launch post.
GDPx leaderboard: Opus 5.5, Gemini 4 Carbon, GPT-6 Astra
| Rank | Model | GDPx score |
|---|---|---|
| 1 | Claude Opus 5.5 | 12.0 |
| 2 | Gemini 4 Carbon | 10.9 |
| 3 | GPT-6 Astra | 9.6 |
| 4 | Muse Wonder Full | 9.0 |
| 5 | Claude Fable 5.1 | 8.5 |
Further down the official leaderboard, GPT 6.1 Sol scored 8.4, DeepSeek 4.1 Flash 1017 scored 5.4, Grok 4.7 scored 3.9 and GLM 5.3 finished last at 2.7. Unite.AI reported average costs per task of $51.41 for Opus 5.5, $53.99 for GPT-6 Astra and $16.29 for Gemini 4 Carbon.
SemiAnalysis called Opus 5.5 "the best all-around model," winning quant trading, hardware design and chip design. GPT-6 Astra led recursive self-improvement and materials science, Gemini 4 Carbon led biology, and Meta's Muse Wonder Full led cybersecurity.
Why safety classifiers matter in agent benchmarks
The firm said "certain models like Opus 5.5 and GPT-6 Astra have sandbagged overall scores due to their safety classifiers," because those filters sometimes block work in cyber, biology and alignment. It published a second ranking without those three domains, which reshuffles the order.
Finance and legal were the hardest areas. "Our finance and legal tasks are so challenging that basically every model scores exactly 0," the authors wrote. On one finance task with 11 checks, no model fully solved it in 25 attempts, and the best run passed 8. On a Series B redline task graded with partial credit, Opus 5.5 averaged 44% and GPT-6 Astra 25%.
Context: a successor to GDPval
SemiAnalysis positions GDPx against OpenAI's GDPval, released Sept. 25, 2025, arguing that its fixed, non-interactive tasks are outdated for agents. "Not including domain-specific, simulated work environments for each task is the single biggest issue with GDPval," the post said, adding that human expert agreement was 71% on its gold subset.
The low ceiling contrasts with coding-only tests such as Replit's ViBench 2.0, where the top agent fully passed 65% of builds, even as models like Claude Opus 5.5 take on longer agentic jobs.
What's next
The site labels the leaderboard v1.0 and "live." Most tasks are private, which limits outside reproduction, and all scores so far were run by SemiAnalysis and its partners.
FAQ
What is GDPx?
A SemiAnalysis benchmark of 206 long-running agent tasks in 11 domains, including chip design, cybersecurity, biology, finance and legal work.
Which AI model leads GDPx?
Claude Opus 5.5, with a score of 12.0, followed by Gemini 4 Carbon at 10.9 and GPT-6 Astra at 9.6.
Why are GDPx scores so low?
Tasks run in simulated work environments for up to eight hours, finance and legal tasks are graded strictly, and safety filters blocked some work in sensitive domains.
References
- GDPx: Measuring Frontier Model Capabilities — SemiAnalysis (Nov. 2, 2026)
- GDPx Leaderboard — SemiAnalysis (v1.0)
- SemiAnalysis Launches GDPx Benchmark Spanning 11 Agentic Domains — Unite.AI (Nov. 2, 2026)
Editor responsible for AI Newsway. Writes about AI, developer tools, cloud and productivity technology.
Comments 0
Comments are currently disabled.