Metraipolis › archive › dc_aa3dd9529fcd

A rigor-matched audit of periodic-step layer skipping for efficient llm inference: conflayers versus swift, with a supplemental analysis of trained routing alternatives — arXiv:2608.28846

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.28846
License (as archived): CC0-1.0
Archived: 2026-10-17T07:47:19.659Z
SHA-256 of the archived text: 54811d5661ebb95a335d65cf76e45700c51f66666bb612f6a82e99c9778deba7
Stable URL of this page: https://metraipolis.com/documento/dc_aa3dd9529fcd

How to cite

A rigor-matched audit of periodic-step layer skipping for efficient llm inference: conflayers versus swift, with a supplemental analysis of trained routing alternatives — arXiv:2608.28846 — arXiv (en), https://arxiv.org/abs/2608.28846 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_aa3dd9529fcd (Metraipolis archive dc_aa3dd9529fcd, sha256 54811d5661ebb95a335d65cf76e45700c51f66666bb612f6a82e99c9778deba7)

BibTeX

@misc{dcaa3dd9529fcd,
  title = {A rigor-matched audit of periodic-step layer skipping for efficient llm inference: conflayers versus swift, with a supplemental analysis of trained routing alternatives — arXiv:2608.28846},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.28846},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_aa3dd9529fcd; sha256 54811d5661ebb95a335d65cf76e45700c51f66666bb612f6a82e99c9778deba7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.28846. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Layer-skipping methods for efficient LLM inference decide, at some granularity, which transformer layers to execute for a given input. We present a rigor-matched, three-seed audit of two periodic-step, search-based methods that make this decision online at inference time and re-evaluate it every few generation steps: a confidence-gated early-exit baseline (ConfLayers) and genuine self-speculative decoding (SWIFT, Xia et al. 2024), together with vanilla autoregressive decoding, across Qwen2.5-0.5B and Qwen2.5-1.5B (Yang et al. 2024) on GSM8K (Cobbe et al. 2021) and CNN/DailyMail (Nallapati et al. 2016; See et al. 2017). SWIFT is strongest on accuracy in three of four cells, while ConfLayers is dominated everywhere, with particularly large deficits on GSM8K at 1.5B. Separating online-search overhead from pure inference cost, we find that SWIFT's true inference speed is faster than ConfLayers's in all four cells (5-21%), reversing the naive wall-clock ranking in three; ConfLayers's search overhead is small and stable (1-2% of cost), whereas SWIFT's is larger and more variable across seeds (up to 28.7%). We additionally examine two trained routing methods, LayerRoute (Sikdar, 2026), a per-sequence, input-conditioned hard gate, and LayerDrop (Fan et al. 2020), a fixed, input-independent pruning pattern, as a supplemental analysis rather than a head-to-head comparison because both operate at a coarser decision granularity. Under a verified protocol with genuine per-input gating, a genuine full-model baseline, and genuine inference-time compute skipping, both show modest real speedups (1.08-1.33x) but accuracy well below the periodic-step methods, including near-total LayerRoute collapse on GSM8K at 1.5B (0.003 mean exact-match across three seeds). We release the full audit protocol as a template for rigor-matched efficiency comparisons.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.28846 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T07:28:26.692Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).