Metraipolis › archive › dc_458eac7b77cc
LoopFold: Parallelizing Recurrent Depth in Looped Transformers — arXiv:2610.35556
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.35556
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-27T08:12:26.490Z
- SHA-256 of the archived text
a8e9bfdd0e0166ab4d5080aaddc85c2249eb24cffd70a0b8490aee4fe64a4a74- Stable URL of this page
https://metraipolis.com/documento/dc_458eac7b77cc
How to cite
LoopFold: Parallelizing Recurrent Depth in Looped Transformers — arXiv:2610.35556 — arXiv (en), https://arxiv.org/abs/2610.35556 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_458eac7b77cc (Metraipolis archive dc_458eac7b77cc, sha256 a8e9bfdd0e0166ab4d5080aaddc85c2249eb24cffd70a0b8490aee4fe64a4a74)
BibTeX
@misc{dc458eac7b77cc,
title = {LoopFold: Parallelizing Recurrent Depth in Looped Transformers — arXiv:2610.35556},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.35556},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_458eac7b77cc; sha256 a8e9bfdd0e0166ab4d5080aaddc85c2249eb24cffd70a0b8490aee4fe64a4a74},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.35556. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Looped Transformers increase computation per token by repeatedly applying a shared backbone, offering a way to improve language modeling and downstream task performance at a fixed backbone parameter count. However, each loop depends on the preceding loop, so increasing recurrent depth also increases decoding latency. Existing approaches include Parallel Loop Transformers, which reduce latency by overlapping loop computation across token positions, but yield rapidly diminishing marginal accuracy gains at larger depths. To address this challenge, we introduce LoopFold, which folds computation at multiple recurrent depths into parallel batches for the same token prefix. A lightweight predictor estimates inputs to later loops so they can start before earlier loops finish. A corrector then refines the outputs of these later loops using newly available results from earlier loops. All components are trained jointly using only the final next-token prediction loss. In experiments with 1.4B models, deeper LoopFold configurations achieve progressively higher average zero-shot accuracy across eight tasks, rising from 53.41% to 56.38%. Vanilla looped models and PLT show limited gains over the same depth range. Compared with a nine-loop vanilla model, LoopFold uses five sequential stages and additional parallel computation to reduce decoding latency by 46%. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.35556 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-27T07:56:00.270Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).