Metraipolis › archive › dc_bd275ba428b5

A decodability criterion predicts when hidden-state selection beats majority voting in large language models — arXiv:2608.17124

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.17124
License (as archived)
CC0-1.0
Archived
2026-10-18T14:28:30.040Z
SHA-256 of the archived text
3d75d1ea737d2ca01d389e1b58c8d296040158fcfdc36ebc5e72de54f101a481
Stable URL of this page
https://metraipolis.com/documento/dc_bd275ba428b5

How to cite

A decodability criterion predicts when hidden-state selection beats majority voting in large language models — arXiv:2608.17124 — arXiv (en), https://arxiv.org/abs/2608.17124 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bd275ba428b5 (Metraipolis archive dc_bd275ba428b5, sha256 3d75d1ea737d2ca01d389e1b58c8d296040158fcfdc36ebc5e72de54f101a481)
BibTeX
@misc{dcbd275ba428b5,
  title = {A decodability criterion predicts when hidden-state selection beats majority voting in large language models — arXiv:2608.17124},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.17124},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bd275ba428b5; sha256 3d75d1ea737d2ca01d389e1b58c8d296040158fcfdc36ebc5e72de54f101a481},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.17124. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Combining the answers a large language model (LLM) samples for a question into one decision is a test-time information fusion problem, usually solved by majority voting. Voting is unreliable on difficult questions, where the sampled answers share correlated errors, so the wrong answer can win and drawing more samples makes the decision worse. Selecting a candidate by reading a correctness signal from the model's hidden states is a promising alternative, but its accuracy varies across models and tasks, and no measure indicates when it can be trusted. In this paper, we propose CASE (Correctness-Axis SElection), a dynamic selection combiner that trains a linear gate on the answer-token hidden state and selects the highest-scoring candidate. Its main contribution is decodability, a leakage-free measure of how well the gate ranks a question's correct candidates above its incorrect ones, which predicts whether hidden-state selection will outperform voting. A conventional probe appears accurate only because of question-identity leakage, which vanishes under question-grouped evaluation. On held-out data, decodability predicts the accuracy gain of selection over voting with a Pearson correlation r=0.75 and a decision threshold near AUC=0.60. Across general and medical LLMs, CASE improves over voting by up to 19 points on medium-difficulty questions and 16.8 points on hard questions. Decodability depends on the aligned knowledge a model must recall, not on its scale, and its prediction transfers to an unseen scientific domain within 3.8 points. It thus provides a practical criterion, measurable in advance for a given model and task, for choosing between learned selection and majority voting.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-19 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.17124 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T14:27:54.236Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-19?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).