Metraipolis › archive › dc_afa41916de1d
Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling — arXiv:2609.19499
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.19499
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-10T03:48:58.926Z
- SHA-256 of the archived text
1d9a6552ee82e937462722c869af3b77ffbc514508f59177cc670ad0b4ab658e- Stable URL of this page
https://metraipolis.com/documento/dc_afa41916de1d
How to cite
Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling — arXiv:2609.19499 — arXiv (en), https://arxiv.org/abs/2609.19499 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_afa41916de1d (Metraipolis archive dc_afa41916de1d, sha256 1d9a6552ee82e937462722c869af3b77ffbc514508f59177cc670ad0b4ab658e)
BibTeX
@misc{dcafa41916de1d,
title = {Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling — arXiv:2609.19499},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.19499},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_afa41916de1d; sha256 1d9a6552ee82e937462722c869af3b77ffbc514508f59177cc670ad0b4ab658e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.19499. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Test-time scaling can improve large language model reasoning by generating and combining multiple candidate responses. In sampling-based methods, the inference budget is often described by the number of generated candidates, N. However, N tells us how many candidates are generated, not how they are executed. The same candidate budget can be produced in one batched generation call or split across several sequential calls with smaller batch sizes. We first study the effect of increasing N on reasoning accuracy using Phi-3-mini and Qwen2.5-1.5B on 500 GSM8K prompts. As expected, increasing N from 1 to 8 improves accuracy by 8.4 percentage points for Phi-3-mini and 18.4 points for Qwen2.5-1.5B. However, accuracy alone does not show the systems cost of using a larger candidate budget. We therefore fix N = 8 and compare four generation schedules: 1x8, 2x4, 4x2, and 8x1, where axb denotes a generation calls with b candidates per call. We measure latency, throughput, GPU-hours, and gross GPU-device energy while keeping the total candidate count fixed. On A100 GPUs, eight serial calls use 4.64-4.86x as much gross GPU-device energy and have 5.77-6.12x the P95 latency of one batched call with eight candidates. The same pattern appears across three independently scheduled A100 nodes per model and in short-output SciQ/V100 experiments. These results show that candidate count alone is not enough to describe the systems cost of multi-candidate test-time scaling. When candidates are independent and memory allows it, fewer generation calls with larger batch sizes are more efficient. Evaluations should therefore report not only candidate count and accuracy, but also generation schedule and GPU-level systems metrics. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-18 in cs.DC). Categorie arXiv: Machine Learning (cs.LG) ; Distributed, Parallel, and Cluster Computing (cs.DC); Performance (cs.PF). Pagina del lavoro: https://arxiv.org/abs/2609.19499 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-10T03:24:19.935Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).