Metraipolis › archive › dc_fc410994579e
Thinking Costs Tokens: When More Structure is Worth the Price — arXiv:2608.27506
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.27506
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T10:47:05.571Z
- SHA-256 of the archived text
ea5efbeaa447d774b1727aabb89f287182ad4779bb7bf9a36610b076a6958c41- Stable URL of this page
https://metraipolis.com/documento/dc_fc410994579e
How to cite
Thinking Costs Tokens: When More Structure is Worth the Price — arXiv:2608.27506 — arXiv (en), https://arxiv.org/abs/2608.27506 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fc410994579e (Metraipolis archive dc_fc410994579e, sha256 ea5efbeaa447d774b1727aabb89f287182ad4779bb7bf9a36610b076a6958c41)
BibTeX
@misc{dcfc410994579e,
title = {Thinking Costs Tokens: When More Structure is Worth the Price — arXiv:2608.27506},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.27506},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fc410994579e; sha256 ea5efbeaa447d774b1727aabb89f287182ad4779bb7bf9a36610b076a6958c41},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.27506. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Adding inference structure to a language model lets it search, verify, and revise, but these actions consume the very budget they are supposed to use well. In this paper, we investigate whether there exists a token-budget threshold, below which the overhead of planning and verification hurts performance and above which it helps. We evaluate two systems on FinQA and TAT-QA financial reasoning tasks, using GPT-5.4 mini across 14 budget tiers ranging from 250 to 42,000 output-equivalent tokens. The first system is a monolith, which is a single LLM call. The second is a verified search architecture that adds planning, label-blind checking, and repair capabilities. We run 1,000 cases for a total of 28,000 completed cells. Both systems score 0% at the two lowest tiers, where neither can fit a complete prompt. At 1,000 tokens, the monolith reaches 18% accuracy while verified search scores near 0%, since the planning overhead leaves no room for an answer. From 1,500 tokens onward, verified search surpasses the monolith and maintains a consistent advantage, reaching approximately 44% at the highest tiers while the monolith reaches approximately 40%. The crossover occurs between 1,000 and 1,500 output-equivalent tokens, confirmed by a strict intersection-union test ($p \le 0.001$ at both endpoints). Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-31 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.27506 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T10:33:42.331Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-31?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).