Metraipolis › archive › dc_6c614141c577
Best-of-Depth: Learning Exploratory Dynamics in Looped Transformers — arXiv:2611.04663
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.04663
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T12:30:28.609Z
- SHA-256 of the archived text
32cb481e4aabe223b0dad185cdf8bc3e71387d91269d62cf137b9f90d4bae75a- Stable URL of this page
https://metraipolis.com/documento/dc_6c614141c577
How to cite
Best-of-Depth: Learning Exploratory Dynamics in Looped Transformers — arXiv:2611.04663 — arXiv (en), https://arxiv.org/abs/2611.04663 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6c614141c577 (Metraipolis archive dc_6c614141c577, sha256 32cb481e4aabe223b0dad185cdf8bc3e71387d91269d62cf137b9f90d4bae75a)
BibTeX
@misc{dc6c614141c577,
title = {Best-of-Depth: Learning Exploratory Dynamics in Looped Transformers — arXiv:2611.04663},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.04663},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6c614141c577; sha256 32cb481e4aabe223b0dad185cdf8bc3e71387d91269d62cf137b9f90d4bae75a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04663. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Looped Transformers extend inference-time computation through repeated latent-state updates, but final-depth training does not directly encourage useful intermediate alternatives. We introduce Best-of-Depth (BoD), which trains a recurrent trajectory as a candidate set by minimizing prediction loss across depths. Different valid targets can be matched at different depths without architectural changes or depth-specific labels. An exact-match analysis connects this objective to solution coverage. On N-Queens completion and graph coloring, BoD improves coverage over final-depth and intermediate-supervision objectives. N-Queens coverage reaches 91.03%, versus 45.75% for final-depth training. Further experiments show discovery beyond the training depth and recovery of solutions withheld from teacher pools on familiar inputs. Fine-tuning a pretrained looped language model with BoD yields higher pooled code-generation success than final-depth training when programs are generated separately at evaluated fixed depths. These results support training recurrent computation for solution discovery along its trajectory. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.04663 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T12:26:47.288Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).