Metraipolis › archive › dc_bbae77139af0

Calibration Is Not Verification: Falsifiability-Aware Conformal Routing for Mixture-of-Agents — arXiv:2609.25959

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.25959
License (as archived)
CC0-1.0
Archived
2026-10-08T13:43:34.959Z
SHA-256 of the archived text
3b6a55232fa58ff7948ac4600f95584af08f3b942336e7d5aa9db47ff89b09aa
Stable URL of this page
https://metraipolis.com/documento/dc_bbae77139af0

How to cite

Calibration Is Not Verification: Falsifiability-Aware Conformal Routing for Mixture-of-Agents — arXiv:2609.25959 — arXiv (en), https://arxiv.org/abs/2609.25959 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bbae77139af0 (Metraipolis archive dc_bbae77139af0, sha256 3b6a55232fa58ff7948ac4600f95584af08f3b942336e7d5aa9db47ff89b09aa)
BibTeX
@misc{dcbbae77139af0,
  title = {Calibration Is Not Verification: Falsifiability-Aware Conformal Routing for Mixture-of-Agents — arXiv:2609.25959},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.25959},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bbae77139af0; sha256 3b6a55232fa58ff7948ac4600f95584af08f3b942336e7d5aa9db47ff89b09aa},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.25959. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multi-agent language systems often treat agreement as evidence, yet heterogeneous agents can jointly repeat an unsupported claim or omit a correct specialist fact. We introduce C-MoA, an agreement-based conformal filter that turns inter-agent semantic support into a claim-level nonconformity score and calibrates a retention threshold at the example level, giving distribution-free within-domain factuality control for heterogeneous Mixture-of-Agents. C-MoA is effective: it nearly doubles retained-claim precision on long-form generation (from 0.41 to 0.75), certifies a human-labelled medical set, and transfers across domains without recalibration; its one failure mode is short-form answering, where consensus is cheap and the score is left near chance. We then ask whether counterfactual falsifiability can push past consensus, and introduce CONTRA-MoA, which adds a blinded near-miss tournament, leave-one-agent-out stability, and availability-aware fusion. This extension helps only where the verifier holds domain knowledge, dropping half of the false medical claims at 0.940 precision, whereas with a memory-only judge the added signals are near chance (AUC 0.531 and 0.511) and naive max fusion degrades the working agreement signal from 0.687 to 0.652. The message is twofold: agreement-based conformal calibration delivers reliable, transferable factuality control, while moving beyond consensus requires a knowledgeable verifier, availability-aware signals, and robust fusion.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.MA).
Categorie arXiv: Multiagent Systems (cs.MA).
Pagina del lavoro: https://arxiv.org/abs/2609.25959 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T13:18:03.287Z dalla pagina https://arxiv.org/catchup/cs.MA/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).