Metraipolis › archive › dc_6a1fb57737ee
Ceiling of a Task: When Can a Transformer Succeed Without Its Chain of Thought? — arXiv:2609.33134
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.33134
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T20:56:26.551Z
- SHA-256 of the archived text
9a2421df429bcb065c4ecce243e568af0eb1c816e09bbcb9958e0cd86b48c0b6- Stable URL of this page
https://metraipolis.com/documento/dc_6a1fb57737ee
How to cite
Ceiling of a Task: When Can a Transformer Succeed Without Its Chain of Thought? — arXiv:2609.33134 — arXiv (en), https://arxiv.org/abs/2609.33134 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6a1fb57737ee (Metraipolis archive dc_6a1fb57737ee, sha256 9a2421df429bcb065c4ecce243e568af0eb1c816e09bbcb9958e0cd86b48c0b6)
BibTeX
@misc{dc6a1fb57737ee,
title = {Ceiling of a Task: When Can a Transformer Succeed Without Its Chain of Thought? — arXiv:2609.33134},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.33134},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6a1fb57737ee; sha256 9a2421df429bcb065c4ecce243e568af0eb1c816e09bbcb9958e0cd86b48c0b6},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33134. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Reasoning models generate long chains of thought before they answer, yet it is debated whether the content of these chains does real computational work or is largely decorative. We study this question by viewing a transformer as a shallow circuit. One forward pass through a fixed number of layers has constant depth, so any procedure that runs the model a constant number of times is a shallow circuit. We call the best accuracy that a shallow circuit can reach on a task the ceiling of the task, and a task is serial if its ceiling lies below one. We prove three results on serial tasks that hold for every transformer, no matter how it was trained. Necessity: replacing the chain by anything that does not depend on its content, such as filler tokens or a restatement of the question, drives the accuracy down to the ceiling, and on a maximally serial task down to chance. Depth: no shallow computation can write the chain of a model whose accuracy exceeds the ceiling, not even approximately. Locality: the answer is one shallow pass away from the finished chain, so all of the serial reasoning happens in the chain. On word problems of finite groups, whose ceilings are known, small transformers trained from scratch, with or without reinforcement learning, attain the predicted numbers: chain-trained models solve every input length and fall to chance when the chain is erased, chainless models collapse to the ceiling as the input length grows, and open-weight reasoning models given the same problem in words return to the baseline without their chain. On MATH-500 and AIME, erasing the chain costs open reasoning models 0.52 to 0.82 accuracy, a sentence shuffle is harmless, and a token shuffle is as harmful as erasing; the same holds for checkpoints trained by GRPO with a correct or a random reward. The ceiling of a task therefore answers when a transformer can succeed without its chain of thought. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.33134 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:29:01.164Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).