Metraipolis › archive › dc_1bf6ab2ed22d
Temperature Fragility and the Conditional Benefits of Truncation Sampling — arXiv:2609.15476
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.15476
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T23:16:06.503Z
- SHA-256 of the archived text
4a0c353afedbeacdf37d275032cab316622f1e8d26d2c8360844b05013fe08f9- Stable URL of this page
https://metraipolis.com/documento/dc_1bf6ab2ed22d
How to cite
Temperature Fragility and the Conditional Benefits of Truncation Sampling — arXiv:2609.15476 — arXiv (en), https://arxiv.org/abs/2609.15476 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1bf6ab2ed22d (Metraipolis archive dc_1bf6ab2ed22d, sha256 4a0c353afedbeacdf37d275032cab316622f1e8d26d2c8360844b05013fe08f9)
BibTeX
@misc{dc1bf6ab2ed22d,
title = {Temperature Fragility and the Conditional Benefits of Truncation Sampling — arXiv:2609.15476},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.15476},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1bf6ab2ed22d; sha256 4a0c353afedbeacdf37d275032cab316622f1e8d26d2c8360844b05013fe08f9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.15476. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models generate text by sampling each token from a predicted distribution, and a temperature parameter sets how far the draw strays from the most probable tokens. Truncation samplers such as top-p and min-p discard the least probable tokens before the draw, so that sampling at high temperature stays coherent. Their reported accuracy gains come from temperatures of 1.5 to 3, while the defaults of deployed systems cluster between 0.6 and 1.0. Whether they change accuracy at those defaults, and for which models, has not been measured. We test thirteen open-weight models on GSM8K and MMLU-Pro at temperatures 0.7, 1.0, and 1.3 in one controlled pipeline, ten of them under eight decoding configurations. Six of the thirteen models lose 17 to 38 accuracy points on MMLU-Pro between 0.7 and 1.3, and the other seven lose at most 10. The lost accuracy comes from generations that run to the token limit or never state an answer. These results suggest that truncation samplers improve accuracy primarily when higher temperatures substantially degrade model performance. Where accuracy remains stable across temperatures, none of the tested truncation samplers improves on plain temperature sampling. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.15476 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T22:51:09.945Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).