Metraipolis › archive › dc_59d2ca5ae161

Allocating Recurrent Compute in Looped Language Models — arXiv:2608.18230

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.18230
License (as archived)
CC0-1.0
Archived
2026-10-18T13:53:10.279Z
SHA-256 of the archived text
e35223226ec0b6c388486a16a10f776b425e09fa00980254a4c6d4697350d962
Stable URL of this page
https://metraipolis.com/documento/dc_59d2ca5ae161

How to cite

Allocating Recurrent Compute in Looped Language Models — arXiv:2608.18230 — arXiv (en), https://arxiv.org/abs/2608.18230 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_59d2ca5ae161 (Metraipolis archive dc_59d2ca5ae161, sha256 e35223226ec0b6c388486a16a10f776b425e09fa00980254a4c6d4697350d962)
BibTeX
@misc{dc59d2ca5ae161,
  title = {Allocating Recurrent Compute in Looped Language Models — arXiv:2608.18230},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.18230},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_59d2ca5ae161; sha256 e35223226ec0b6c388486a16a10f776b425e09fa00980254a4c6d4697350d962},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.18230. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Looped language models improve reasoning and knowledge manipulation by applying shared computation repeatedly. Existing systems usually repeat an entire layer stack, although a mixer and a dense feed-forward network (FFN) perform different operations and have different costs. We ask a narrower question: what should loop? We view recurrence as repeated composition of a state update and argue that an application is valuable when it exposes a new cross-position influence direction that remains observable at the task readout. Iterative Transport Rank (ITR) describes the cumulative influence trajectory; marginal ITR describes the nonredundant influence contributed by successive applications. This view motivates MixerLoop, which repeats each Gated DeltaNet mixer while applying its dense FFN once. We compare MixerLoop with no recurrence and full-block recurrence at 15M and 110M parameters under the same data, initialization, and architecture. A finite context-off intervention tests whether later mixer applications produce distinct, non-negligible, and beneficial changes at the final language-model readout. MixerLoop surpasses FullLoop on aggregate CORE at 15M and retains 41.5% of its CORE improvement at 110M while reducing recurrent-backbone projection FLOPs by 45.9%. These results show that the benefits of recurrent depth can be retained without repeatedly executing the dense FFN.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-20 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.18230 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T13:50:58.893Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-20?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).