Metraipolis › archive › dc_ac6210acfcc2

Library of Layers (LoL): Adaptively Ordering Transformer Layers at Runtime — arXiv:2611.03526

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.03526
License (as archived)
CC0-1.0
Archived
2026-11-03T18:21:30.013Z
SHA-256 of the archived text
efe8978d99423d232290a54a09b6402d95f222fbe73277a2004080157e945c57
Stable URL of this page
https://metraipolis.com/documento/dc_ac6210acfcc2

How to cite

Library of Layers (LoL): Adaptively Ordering Transformer Layers at Runtime — arXiv:2611.03526 — arXiv (en), https://arxiv.org/abs/2611.03526 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ac6210acfcc2 (Metraipolis archive dc_ac6210acfcc2, sha256 efe8978d99423d232290a54a09b6402d95f222fbe73277a2004080157e945c57)
BibTeX
@misc{dcac6210acfcc2,
  title = {Library of Layers (LoL): Adaptively Ordering Transformer Layers at Runtime — arXiv:2611.03526},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.03526},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ac6210acfcc2; sha256 efe8978d99423d232290a54a09b6402d95f222fbe73277a2004080157e945c57},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03526. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Computer programmers chain together the same reusable computational primitives, for example, conditional statements or loops, but in different orders to implement different algorithms. In contrast, existing language model architectures always execute their layers in the same order. We introduce Library of Layers (LoL), an architecture that dynamically orders its layers during inference to solve the problem at hand. LoL has a single hyperparameter, the size of its global library of transformer blocks, that allows it to transition seamlessly between parameter reuse and sparsity, generalizing the parameter-efficiency and FLOPs-efficiency tradeoffs of dense, depth-recurrent, and mixture of experts architectures in one unified family. We show that LoL achieves competitive performance with all three above baselines when pretrained on text data. On synthetic sudoku and maze data, we observe that the model employs distinct layer orderings to implement different solution strategies, and we can intervene on routing to force a model to apply a specific solution strategy to a problem instance. In a multilingual natural language setting, we find that LoL chooses different layer order patterns based on language and script. Similarly, we train LoL on safety data and fit probes to its router outputs, finding that these probes distinguish safe from unsafe text with high AUROC. LoL provides a path toward a future in which models dynamically compose their own computational graph, enabling more efficient and controllable reasoning.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2611.03526 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:09:53.758Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).