AI Teammates Hit Best Deal in 14% of Disputes

Researchers from Stanford University, the University of Oxford and MIT on Monday, Nov. 2, released the Collaborative Intelligence Benchmark, a 240-task test of how well AI models work as teammates. None of the 18 models tested came close to a perfect score, and the flagship models from Google, OpenAI and Anthropic reached the best overall deal in only 14% of conflict tests.

Key takeaways

• The benchmark scores 18 closed- and open-weight models on 240 tasks across eight teamwork skills; the best result was 82.2 out of 100.
• Frontier models almost always refused unsafe shortcuts and reported misconduct, but passed only 45% of emotional-intelligence tests.
• Models with nearly the same general capability differed by up to 11 points as teammates.

What the Collaborative Intelligence Benchmark measures

The paper, posted to arXiv by Sean Wu, James Zou and seven co-authors, argues that AI progress is usually measured by what one model does alone, even as agents increasingly work with people and with each other. The benchmark borrows from decades of research on human teams.

Its eight dimensions are resisting unsafe pressure, reporting misconduct, sharing and seeking information, emotional intelligence, keeping the team on track, handling conflict, taking initiative and taking over a compromised session. Each dimension has 30 tasks: 20 public and 10 held out for evaluation.

Scenarios are built from real records, including NTSB and NASA safety reports, clinical case reports, FDA and research-misconduct findings, negotiation datasets and coding benchmarks. Every task has an outcome that can be checked by code. To separate teamwork from raw skill, a model is scored only on tasks it can solve alone when given full information.

AI teamwork scores: Gemini 4 Argon, GPT-6 Astra, Claude Opus 5.5

At default reasoning settings, the paper's main table puts Google's Gemini 4 Argon at 76.6, OpenAI's GPT-6 Astra at 76.5, GPT-6.1 Sol at 74.5 and Anthropic's Claude Opus 5.5 at 71.0. The top score of 82.2 came from Gemini 4 Argon at high reasoning effort. OpenAI's open-weight gpt-oss-120b finished last at 19.1.

The project's GitHub leaderboard, which lists 12 models, currently shows GPT-6 Astra first at 78.0, so rankings near the top depend on the version and settings used.

Where AI agents fail as teammates

The clearest weakness was pressure from a teammate. "Frontier models rarely give in to unsafe pressure, but they struggle to hold firm in a conflict," the authors wrote. In emotional-intelligence tests, the frontier group passed only 45%.

Inherited work was another gap. The same models passed 81% of tasks when taking over after a weaker model had cheated, against 97% when they started those tasks themselves.

Why it matters for AI agents at work

The results suggest that leaderboards built on solo tasks miss how agents behave in shared workflows. That echoes other recent agent tests, such as SemiAnalysis's GDPx, where even Claude Opus 5.5 completed only a small share of long, realistic jobs.

What's next

The code is released under the MIT license and most task data under CC BY 4.0, with seven scenarios under a non-commercial license. The 80 held-out tasks stay private, which lets the authors test new models without the answers leaking into training data.

FAQ

What is the Collaborative Intelligence Benchmark?
A 240-task test from Stanford, Oxford and MIT researchers that scores how well AI models act as teammates across eight skills, from refusing unsafe shortcuts to handling conflict.

Which AI model is the best teammate?
In the paper, Gemini 4 Argon scored highest, 82.2 at high reasoning effort and 76.6 at default, narrowly ahead of GPT-6 Astra at 76.5.

Can developers run the benchmark?
Yes. The 160 public tasks and evaluation code are on GitHub; the 80 held-out tasks are kept private.

References

• Measuring AI Agents as Teammates with Collaborative Intelligence — arXiv (Nov. 2, 2026)
• zou-group/collaborative-intelligence — GitHub
• Collaborative Intelligence project page — Zou Group, Stanford University