Metraipolis › archive › dc_c2ca34427f40
Not All Loops Cost the Same: Phase-Dependent Depth for Serving Looped Transformers — arXiv:2611.07211
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.07211
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-06T14:43:26.687Z
- SHA-256 of the archived text
ef3fb3ed69e4ca1005fdeed5198db6a5fba0683f2e72e99267d15d1ac864d183- Stable URL of this page
https://metraipolis.com/documento/dc_c2ca34427f40
How to cite
Not All Loops Cost the Same: Phase-Dependent Depth for Serving Looped Transformers — arXiv:2611.07211 — arXiv (en), https://arxiv.org/abs/2611.07211 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c2ca34427f40 (Metraipolis archive dc_c2ca34427f40, sha256 ef3fb3ed69e4ca1005fdeed5198db6a5fba0683f2e72e99267d15d1ac864d183)
BibTeX
@misc{dcc2ca34427f40,
title = {Not All Loops Cost the Same: Phase-Dependent Depth for Serving Looped Transformers — arXiv:2611.07211},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.07211},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c2ca34427f40; sha256 ef3fb3ed69e4ca1005fdeed5198db6a5fba0683f2e72e99267d15d1ac864d183},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.07211. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Looped transformers scale inference-time reasoning by repeatedly reusing a shared core, increasing computation without increasing parameter count. Serving prefilling and decoding with a uniform loop budget couples their costs. We present LoopDuet, a phase-dependent depth and training-free serving approach that selects the two budgets jointly while preserving decoding depth at prefilling boundaries. Our vLLM implementation gathers tokens requiring further recurrence into compact batches, turning the selected allocation into executable savings with existing weights and attention kernels. At runtime, a queue-age controller adjusts prefilling depth to retain quality as demand changes. Experiments on the full HotpotQA dataset with one A100-80GB demonstrate two benefits of this flexibility. First, separate phase budgets improve the quality--throughput trade-off: Huginn delivers 19\% more throughput than the fastest tested uniform setting meeting the same accuracy criterion, while a higher-throughput operating point gains 6.04 EM points over uniform depth eight at 97\% of its throughput. Second, the same execution mechanism retains more quality under load: on two evaluated OLMo arrival traces, adaptive prefilling gains 0.92--1.35 EM points over the most accurate tested fixed-depth controls at the same 100\% attainment of a one-second time-to-first-token target. Together, these results establish LoopDuet as a practical way to turn flexible model computation into higher serving capacity and better quality under load. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC). Pagina del lavoro: https://arxiv.org/abs/2611.07211 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-06T14:19:12.617Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).