Metraipolis › archive › dc_905f16f56510
vSkipper: Translating Dynamic Layer Skipping into LLM Serving Gains — arXiv:2609.37062
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.37062
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T16:59:08.940Z
- SHA-256 of the archived text
edd89d9ed58c4ba0f939c08d3130a3b79fd65206852db2b7058d12dc0372f08b- Stable URL of this page
https://metraipolis.com/documento/dc_905f16f56510
How to cite
vSkipper: Translating Dynamic Layer Skipping into LLM Serving Gains — arXiv:2609.37062 — arXiv (en), https://arxiv.org/abs/2609.37062 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_905f16f56510 (Metraipolis archive dc_905f16f56510, sha256 edd89d9ed58c4ba0f939c08d3130a3b79fd65206852db2b7058d12dc0372f08b)
BibTeX
@misc{dc905f16f56510,
title = {vSkipper: Translating Dynamic Layer Skipping into LLM Serving Gains — arXiv:2609.37062},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.37062},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_905f16f56510; sha256 edd89d9ed58c4ba0f939c08d3130a3b79fd65206852db2b7058d12dc0372f08b},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.37062. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Dynamic layer skipping reduces LLM computation by allowing each token to execute only a subset of the model's layers. However, existing skippers rely on specialized generation loops and do not integrate with modern serving engines. As a result, fewer executed layers do not necessarily translate into lower serving latency: FlexiDepth skips 8 of Llama-3-8B's 32 layers on average, yet its standard generation loop decodes 14.6--21.0% more slowly than the base model. We present vSkipper, a virtualization layer that makes dynamic layer skippers pluggable in serving engines while preserving continuous batching, fixed-shape batches, paged KV caching, and captured decode graphs. At each routed layer, vSkipper groups tokens by the skipper's decision and uses routed execution only when predicted to be profitable. We implement vSkipper in SGLang and evaluate the released FlexiDepth checkpoint against upstream SGLang under identical prompts, arrivals, output lengths, and launch settings. At the knee of upstream's load curve, vSkipper reduces mean end-to-end latency by 36.8% on GSM8K and 13.6% on BBH. Under saturation, it increases request throughput by 11.3% and 7.4%. Serving adds no statistically resolved quality loss beyond the checkpoint's own. Across synthetic skip policies, two Qwen3 skippers, and three GPUs, we demonstrate reuse without workload-specific tuning. To our knowledge, vSkipper is the first system to realize serving-efficiency gains from per-token interior layer skipping within a modern LLM serving engine. The code is open-sourced as an SGLang fork at this https URL Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-30 in cs.DC). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.37062 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T16:31:32.455Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).