Metraipolis › archive › dc_b0ded40237cb

Scute: Exact Batched Serving for PEFT Adapters Beyond LoRA — arXiv:2611.01089

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.01089
License (as archived)
CC0-1.0
Archived
2026-11-03T18:36:42.015Z
SHA-256 of the archived text
d869aa6da86f480870bea99c32a11ed2cef90ae1091396405a90f75ba410afe0
Stable URL of this page
https://metraipolis.com/documento/dc_b0ded40237cb

How to cite

Scute: Exact Batched Serving for PEFT Adapters Beyond LoRA — arXiv:2611.01089 — arXiv (en), https://arxiv.org/abs/2611.01089 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b0ded40237cb (Metraipolis archive dc_b0ded40237cb, sha256 d869aa6da86f480870bea99c32a11ed2cef90ae1091396405a90f75ba410afe0)
BibTeX
@misc{dcb0ded40237cb,
  title = {Scute: Exact Batched Serving for PEFT Adapters Beyond LoRA — arXiv:2611.01089},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01089},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b0ded40237cb; sha256 d869aa6da86f480870bea99c32a11ed2cef90ae1091396405a90f75ba410afe0},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01089. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Existing serving systems for large language models batch requests for many different LoRA adapters on one GPU. However, these systems reject adapters from other parameter-efficient fine-tuning (PEFT) methods such as DoRA, (IA), VeRA and OFT. One solution to this is to convert these adapters to LoRA, but this induces a significant memory and accuracy cost. Instead, we show that, under some modifications, a batched LoRA kernel can serve all of these methods exactly and efficiently. We achieve this by writing each method in one normal form, a LoRA computation with six slots for an adapter's own vectors and matrices. Some slots can merge into an adapter's weights at load time; the others remain live and induce a cost. We give ways to efficiently pay this cost and predict it from a zero-parameter byte model. We package this system, Scute, and build it on top of vLLM. Scute serves all five methods, coming as checkpoints from the Hugging Face PEFT library, in one batch, and it matches that library with a mean error of at most 1.5% of each adapter's effect. Under a burst of requests on an RTX A6000, a mix of these methods takes 5-17% more decode time than the same number of LoRA adapters. Furthermore, using Scute, the same GPU can serve 64 to 1024 (IA) and VeRA adapters at 1.5x to at least 2.5x the request rate of their LoRA conversions.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.01089 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:12:54.476Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).