Metraipolis › archive › dc_07b6154239e8

What Confidence Routing Is Actually Doing: Auditing Routing, Calibration, and Commitment in Multi-Agent Deliberation — arXiv:2609.27822

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.27822
License (as archived)
CC0-1.0
Archived
2026-10-08T08:57:20.013Z
SHA-256 of the archived text
a512561e17534fcbd909b8bfecd727788f3433fd527cce2b5bfbfd38638e64ac
Stable URL of this page
https://metraipolis.com/documento/dc_07b6154239e8

How to cite

What Confidence Routing Is Actually Doing: Auditing Routing, Calibration, and Commitment in Multi-Agent Deliberation — arXiv:2609.27822 — arXiv (en), https://arxiv.org/abs/2609.27822 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_07b6154239e8 (Metraipolis archive dc_07b6154239e8, sha256 a512561e17534fcbd909b8bfecd727788f3433fd527cce2b5bfbfd38638e64ac)
BibTeX
@misc{dc07b6154239e8,
  title = {What Confidence Routing Is Actually Doing: Auditing Routing, Calibration, and Commitment in Multi-Agent Deliberation — arXiv:2609.27822},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.27822},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_07b6154239e8; sha256 a512561e17534fcbd909b8bfecd727788f3433fd527cce2b5bfbfd38638e64ac},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.27822. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

A common multi-agent design asks agents to report confidence and lets the highest-scoring agent speak next, implicitly using one scalar both to route the conversation and to estimate uncertainty. We audit this confidence-routed broadcast protocol by separating three trace-level questions: whether it selects the right candidate (routing), whether reported confidence behaves like a probability (calibration), and whether the selected agent publicly states the answer that won the turn (commitment). Our primary study covers 4,181 gpt-oss-120b olympiad-math traces; we repeat the audit on a 2-by-2 actor-by-benchmark grid that adds gemma-4-31B-it and a biology multiple-choice benchmark. In the primary cell, confidence discriminates correct from wrong candidates (AUROC 0.72) but is strongly overconfident (79% mean stated confidence versus 52% accuracy). A cross-fitted, tier-stratified isotonic procedure reduces Expected Calibration Error from 0.278 to 0.008 on held-out candidates, but it does not recover missing discrimination: raw AUROC is only 0.537 and 0.440 in the two Gemma cells. Routing is likewise setting-dependent. Fixed routers differ by at most 1.1 percentage points on gpt-oss/math, whereas raw-confidence argmax performs 5.6 and 11.2 points below random-valid selection in the Gemma cells. Commitment is distinct again: in the primary cell, poll and spoken answers diverge in 20.4% of valid pairs, 62.4% of those revisions are fresh generations, and the unconditional correctness shift is -1.7 points; the other three cells instead range from +0.9 to +12.2 points. The transferable lesson is procedural: routing discrimination, probability calibration, and public commitment must be measured separately before raw confidence is used for deployment decisions.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-24 in cs.MA).
Categorie arXiv: Multiagent Systems (cs.MA) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.27822 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T08:03:18.929Z dalla pagina https://arxiv.org/catchup/cs.MA/2026-09-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).