Metraipolis › archive › dc_dd35008e674e

When Self-Consistency Hurts: The Hidden Cost of Majority Voting in Small Language Models — arXiv:2611.11201

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.11201
License (as archived)
CC0-1.0
Archived
2026-11-10T18:41:20.230Z
SHA-256 of the archived text
e04181acb3663007b3ef7dc29a5fdf1fdecd7f8c8650e781b43a7e977be5fb12
Stable URL of this page
https://metraipolis.com/documento/dc_dd35008e674e

How to cite

When Self-Consistency Hurts: The Hidden Cost of Majority Voting in Small Language Models — arXiv:2611.11201 — arXiv (en), https://arxiv.org/abs/2611.11201 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dd35008e674e (Metraipolis archive dc_dd35008e674e, sha256 e04181acb3663007b3ef7dc29a5fdf1fdecd7f8c8650e781b43a7e977be5fb12)
BibTeX
@misc{dcdd35008e674e,
  title = {When Self-Consistency Hurts: The Hidden Cost of Majority Voting in Small Language Models — arXiv:2611.11201},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.11201},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dd35008e674e; sha256 e04181acb3663007b3ef7dc29a5fdf1fdecd7f8c8650e781b43a7e977be5fb12},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.11201. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We study prompted chain-of-thought (CoT) with self-consistency (SC), i.e. sampling N traces and plurality-voting, on small language models, a regime where per-query compute budgets are tight yet SC is commonly applied with a fixed sample budget inherited from large-model practice, regardless of instance difficulty. Across two model families (Qwen-2.5-0.5B, Gemma-3-1B) at FP16 and MLX-Q4 (4-bit quantisation in Apple's MLX framework), on GSM8K and MATH-500, we run 8 per-task and 4 task-pooled experiments at n=200-400 with 1,000x paired-bootstrap confidence intervals (CIs). Our first result is an upper bound: an oracle gate deciding per prompt whether to invoke SC reaches 109-174% of always-SC's accuracy at 23-30% of its compute in all 12/12 cases, so at N=5 always-SC is dominated everywhere on our grid. That margin is exactly the rate at which SC flips correct greedy answers to wrong, here 3-10.5% of prompts, which on Qwen-Q4 x MATH-500 exceeds the wrong-to-right rate and leaves always-SC 3 pt worse than always-greedy. Our second result is a deployable approximation: a gate over greedy-pass signals skips 45-99% of SC calls, with bootstrap CI half-widths under 5 pt, removing 36-79% of always-SC compute. Accuracy retention at those operating points has point estimates of 98.1-98.8%, which we report as point estimates because their CIs are wide (lower bounds 76-86%) over an SC accuracy of only 15-36%. The evidence covers N=5 on two <=1B families and two math benchmarks, which is the scope we claim. Code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.11201 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:37.510Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).