Metraipolis › archive › dc_9dce289d784d

Improving Test-Time Scaling with Adaptive Looped Transformers — arXiv:2609.35748

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.35748
License (as archived)
CC0-1.0
Archived
2026-10-07T21:53:48.990Z
SHA-256 of the archived text
05388fec45a85f7805076286723c0ae4409e516ddf8aaf299c6fc00ae207b150
Stable URL of this page
https://metraipolis.com/documento/dc_9dce289d784d

How to cite

Improving Test-Time Scaling with Adaptive Looped Transformers — arXiv:2609.35748 — arXiv (en), https://arxiv.org/abs/2609.35748 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_9dce289d784d (Metraipolis archive dc_9dce289d784d, sha256 05388fec45a85f7805076286723c0ae4409e516ddf8aaf299c6fc00ae207b150)
BibTeX
@misc{dc9dce289d784d,
  title = {Improving Test-Time Scaling with Adaptive Looped Transformers — arXiv:2609.35748},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.35748},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_9dce289d784d; sha256 05388fec45a85f7805076286723c0ae4409e516ddf8aaf299c6fc00ae207b150},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35748. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Looped transformers have demonstrated promising parameter efficiency by reusing layers for latent computation. Prior studies compare looped and non-looped models at matched parameters or per-token FLOPs. However, to the best of our knowledge, whether looping improves test-time scaling as outputs grow longer remains underexplored. Through post-training looped transformers, we study the accuracy-compute slope, measured as the accuracy gain per doubling of test-time decoding FLOPs. We find that existing looped transformers often yield steeper slopes than their non-looped baseline, yet underperform it at matched compute. While fixed-depth looping spends extra iterations on every token, our analysis shows that many tokens do not benefit from extra iterations. We therefore propose TaH2, which enables the model to focus extra iterations on the tokens that benefit from looping. It jointly post-trains the backbone and an iteration decider through lookahead depth supervision, which uses online labels indicating whether further iteration improves the prediction. TaH2 improves both the efficiency and attainable accuracy of test-time scaling. On challenging AIME benchmarks, TaH2 improves the accuracy-compute slope by 53% (2.74 vs. 1.79) over the non-looped baseline, exceeding the baseline's peak accuracy by about 3.4 points at matched test-time compute. As the maximum iteration depth increases, existing looped models largely plateau, while TaH2's gain over the non-looped baseline continues to grow from +2.8 points at depth 2 to +3.9 points at depth 8. Our code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.35748 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T21:27:08.706Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).