Metraipolis › archive › dc_ac588fe04386
More Correct Mass, Worse Answers: Why Power Sampling Can Fail and How to Fix It — arXiv:2608.14420
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.14420
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T02:30:41.042Z
- SHA-256 of the archived text
0db307bc4dcab364541942c0d57bd9ce19eba3b699de797e7df444048603ef1e- Stable URL of this page
https://metraipolis.com/documento/dc_ac588fe04386
How to cite
More Correct Mass, Worse Answers: Why Power Sampling Can Fail and How to Fix It — arXiv:2608.14420 — arXiv (en), https://arxiv.org/abs/2608.14420 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ac588fe04386 (Metraipolis archive dc_ac588fe04386, sha256 0db307bc4dcab364541942c0d57bd9ce19eba3b699de797e7df444048603ef1e)
BibTeX
@misc{dcac588fe04386,
title = {More Correct Mass, Worse Answers: Why Power Sampling Can Fail and How to Fix It — arXiv:2608.14420},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.14420},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ac588fe04386; sha256 0db307bc4dcab364541942c0d57bd9ce19eba3b699de797e7df444048603ef1e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14420. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Power Sampling sharpens a language model's distribution over complete generation trajectories, offering a verifier-free way to improve reasoning at inference time. It also has the potential to serve as a general-purpose front end for a broad range of downstream sampling methods. However, we uncover a striking paradox: Power Sampling can drive more probability mass toward correct trajectories while degrading the downstream inference it is intended to enhance. Using self-consistency as a representative case, we observe accuracy drops of up to 18.5 percentage points across models and reasoning benchmarks. We trace this paradox to two mismatches. Dose mismatch arises because a fixed exponent induces drastically different amounts of distributional change across problems. Coverage mismatch arises because global sharpening concentrates mass on a narrow set of dominant paths: high pass@k, often interpreted as evidence of preserved diversity, can therefore coexist with the loss of broad reasoning-path support required for downstream aggregation, search, and selection. Guided by this diagnosis, we replace uniform trajectory exponentiation with a deformation-controlled, support-preserving Power target that calibrates sharpening across problems while limiting the suppression of moderate-probability paths. In a same-budget instantiation with weighted self-consistency, the repaired sampler reverses the losses caused by global Power and outperforms standard multi-sample inference across reasoning benchmarks. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-17 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.14420 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T02:26:17.808Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).