Metraipolis › archive › dc_993be0f5713d

When Multiple Choices Are Correct: Asymmetric Probability Allocation in Jev and Other Decision Models — arXiv:2610.29217

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.29217
License (as archived)
CC0-1.0
Archived
2026-10-23T15:21:12.126Z
SHA-256 of the archived text
a6bfe26154e90afa9f1e135cd4bc00ecc4bc154f0c3475ac79c268fa08d095fd
Stable URL of this page
https://metraipolis.com/documento/dc_993be0f5713d

How to cite

When Multiple Choices Are Correct: Asymmetric Probability Allocation in Jev and Other Decision Models — arXiv:2610.29217 — arXiv (en), https://arxiv.org/abs/2610.29217 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_993be0f5713d (Metraipolis archive dc_993be0f5713d, sha256 a6bfe26154e90afa9f1e135cd4bc00ecc4bc154f0c3475ac79c268fa08d095fd)
BibTeX
@misc{dc993be0f5713d,
  title = {When Multiple Choices Are Correct: Asymmetric Probability Allocation in Jev and Other Decision Models — arXiv:2610.29217},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.29217},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_993be0f5713d; sha256 a6bfe26154e90afa9f1e135cd4bc00ecc4bc154f0c3475ac79c268fa08d095fd},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.29217. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Decision models return normalized probability distributions over candidate choices, which downstream applications can use to select among choices. Although their providers emphasize the reliability of these probabilities, calibration on tasks with a single correct choice does not establish their reliability when multiple choices are correct. We evaluate three decision models (Jev, Liquid d1, and GPT-6 Luna Decisions) on three diverse datasets (MetaTool, HotpotQA, and ECtHR) to study probability allocation and thresholding. Even when models rank every correct choice above every distractor, in 28% to 50% of those lists, at least one correct choice receives a probability below the uniform probability of 1/n, where n is the number of choices. We also present each model with five identical copies of a correct choice and observe probability disparities, with mean gaps ranging from 17 to 79 percentage points. We further show how these allocation disparities affect the recovery of multiple correct choices under probability thresholding. Decision-model probabilities can rank correct choices, but a probability threshold does not identify the set of correct choices.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.29217 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-23T15:09:38.374Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).