Metraipolis › archive › dc_8a66282bb90a
Thinking Hard, Not Smart: Reasoning Models Fail to Ration Test-Time Compute Across Questions — arXiv:2608.07968
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.07968
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T08:49:50.537Z
- SHA-256 of the archived text
580a8f25703e58f2a11663d671b85771d55306c5c96dd9c55f7892ac184e4c1e- Stable URL of this page
https://metraipolis.com/documento/dc_8a66282bb90a
How to cite
Thinking Hard, Not Smart: Reasoning Models Fail to Ration Test-Time Compute Across Questions — arXiv:2608.07968 — arXiv (en), https://arxiv.org/abs/2608.07968 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8a66282bb90a (Metraipolis archive dc_8a66282bb90a, sha256 580a8f25703e58f2a11663d671b85771d55306c5c96dd9c55f7892ac184e4c1e)
BibTeX
@misc{dc8a66282bb90a,
title = {Thinking Hard, Not Smart: Reasoning Models Fail to Ration Test-Time Compute Across Questions — arXiv:2608.07968},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.07968},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8a66282bb90a; sha256 580a8f25703e58f2a11663d671b85771d55306c5c96dd9c55f7892ac184e4c1e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.07968. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Reasoning language models increasingly use test-time compute to improve performance, but existing evaluations typically study this compute one question at a time. Yet when multiple problems share an end-to-end cost or latency constraint, models must decide how to divide limited inference compute among them. We introduce an exam-style evaluation framework for studying this setting, in which a model must distribute one shared token budget across questions with different difficulty and point values to maximize its total score. Across several open and frontier reasoning models, we find that models fail to allocate a shared budget strategically across questions of varying difficulties and values. Models behave largely as greedy sequential solvers: they prioritize questions by presentation order, front-load effort on early questions, and remain insensitive to value, with these tendencies becoming more pronounced as the number of questions grows. Explicit planning prompts spread compute more evenly but do not produce value- or difficulty-aware prioritization. The same behavioral pattern extends from mathematical to code reasoning. These findings establish global budget allocation as a distinct capability that is not captured by conventional per-question evaluation and remains a challenge for current reasoning models. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.07968 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T08:49:41.839Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).