Metraipolis › archive › dc_fb40ccf58fe7

Recurrent Self-Improvement: Dynamic Cross-Loop On-Policy Distillation for Looped Language Models — arXiv:2610.10623

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.10623
License (as archived)
CC0-1.0
Archived
2026-10-09T06:13:36.457Z
SHA-256 of the archived text
eeb9f84877e1681ac6a8d06bbeb57d3fe86e298bc0d2f67399f11f70d712cba1
Stable URL of this page
https://metraipolis.com/documento/dc_fb40ccf58fe7

How to cite

Recurrent Self-Improvement: Dynamic Cross-Loop On-Policy Distillation for Looped Language Models — arXiv:2610.10623 — arXiv (en), https://arxiv.org/abs/2610.10623 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fb40ccf58fe7 (Metraipolis archive dc_fb40ccf58fe7, sha256 eeb9f84877e1681ac6a8d06bbeb57d3fe86e298bc0d2f67399f11f70d712cba1)
BibTeX
@misc{dcfb40ccf58fe7,
  title = {Recurrent Self-Improvement: Dynamic Cross-Loop On-Policy Distillation for Looped Language Models — arXiv:2610.10623},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.10623},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fb40ccf58fe7; sha256 eeb9f84877e1681ac6a8d06bbeb57d3fe86e298bc0d2f67399f11f70d712cba1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.10623. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Looped Language Models (LoopLMs) offer a parameter efficient approach to scaling reasoning by reusing shared parameters across recurrent computation steps. Despite their promise, effective post-training of LoopLMs remains challenging. Existing approaches either provide reward based supervision that is sparse or costly to extend across loops, or rely on external teachers or privileged information, leading to limited teacher availability or teacher-student context mismatch. To address these limitations, we introduce LoopOPD, a cross-loop on-policy distillation framework that uses additional recurrent computation within a LoopLM as its own source of supervision. LoopOPD uses a frozen terminal loop policy as a compute privileged teacher for an intermediate loop student on student generated rollouts, providing dense supervision without an external teacher or privileged information. We further propose Dynamic LoopOPD (D-LoopOPD), which continually refreshes the terminal loop teacher as the shared model parameters are updated, enabling recurrent self-improvement. We characterize how distillation updates propagate across loop depths and derive sufficient conditions under which a single update yields simultaneous local improvement at both loop depths. Experiments on Ouro-Thinking models show that LoopOPD improves mathematical reasoning, while D-LoopOPD yields further gains through dynamic teacher updates. Despite being trained only on mathematical data, the resulting models also improve on general reasoning and code generation benchmarks, demonstrating that recurrent computation can serve as an effective source of supervision for LoopLMs. Our code and model checkpoints will be released upon acceptance.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.10623 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T05:47:15.408Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).