Metraipolis › archive › dc_fe12648dcd86

Measuring AI Agents as Teammates with Collaborative Intelligence — arXiv:2611.02633

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.02633
License (as archived): CC0-1.0
Archived: 2026-11-03T18:14:02.974Z
SHA-256 of the archived text: 269d257d1face24c2f38af184421f27532db214e300aea4ae2f66536f5fb6ba4
Stable URL of this page: https://metraipolis.com/documento/dc_fe12648dcd86

How to cite

Measuring AI Agents as Teammates with Collaborative Intelligence — arXiv:2611.02633 — arXiv (en), https://arxiv.org/abs/2611.02633 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fe12648dcd86 (Metraipolis archive dc_fe12648dcd86, sha256 269d257d1face24c2f38af184421f27532db214e300aea4ae2f66536f5fb6ba4)

BibTeX

@misc{dcfe12648dcd86,
  title = {Measuring AI Agents as Teammates with Collaborative Intelligence — arXiv:2611.02633},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02633},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fe12648dcd86; sha256 269d257d1face24c2f38af184421f27532db214e300aea4ae2f66536f5fb6ba4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02633. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Progress in AI is often measured by what a single model can do on its own. Yet teams of AI agents are starting to make breakthroughs and discoveries in science, medicine, mathematics and software engineering. As AI begins to work in teams of agents and people, the ability to collaborate becomes a core part of their intelligence. Research on human teams has measured this ability for decades. Inspired by this, we introduce the Collaborative Intelligence Benchmark, which measures how well an AI agent works as a teammate. It contains 240 tasks across 8 dimensions of teamwork, from refusing an unsafe shortcut to checking the work that a weaker model leaves behind. Each task comes from a real record or an existing benchmark, from aviation accident reports and clinical cases to coding tasks and negotiations, and has a programmatically verifiable outcome. To isolate teamwork from individual capability, models are scored only on tasks that they can solve by themselves when they have full information. We evaluate 18 closed- and open-weight models and find that every model, across sizes and families, has clear weaknesses as a teammate, with the best model scoring only 82.2 out of 100. Frontier models almost always refuse unsafe shortcuts and report misconduct, yet they cave under interpersonal pressure, reaching the best overall deal in only 14% of conflicts, and their pass rate drops from 97% to 81% when they take over a task in which a weaker model has cheated. Crucially, general capability does not guarantee good teamwork, as models with nearly the same capability score differ by up to 11 points as teammates. The benchmark includes 160 public tasks with code and data, along with 80 held-out tasks for evaluation. As AI agents increasingly work alongside other agents and people, the ability to collaborate effectively is becoming a crucial capability to measure and develop.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.02633 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:09:53.400Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).