Metraipolis › archive › dc_0627fad4fe01

From Overloaded to Guaranteed: High-Throughput Multi-SLO Enforcement for LoRA-Assisted On-Premise LLM Deployment — arXiv:2610.04956

Source
arXiv
Original
https://arxiv.org/abs/2610.04956
License (as archived)
CC0-1.0
Archived
2026-10-06T05:28:10.978Z
SHA-256 of the archived text
e55321a577e023355f042925bc5ff511d229554f76647fed55ad8120ef4819d2
Stable URL of this page
https://metraipolis.com/documento/dc_0627fad4fe01

How to cite

From Overloaded to Guaranteed: High-Throughput Multi-SLO Enforcement for LoRA-Assisted On-Premise LLM Deployment — arXiv:2610.04956 — arXiv, https://arxiv.org/abs/2610.04956 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0627fad4fe01 (Metraipolis archive dc_0627fad4fe01, sha256 e55321a577e023355f042925bc5ff511d229554f76647fed55ad8120ef4819d2)
BibTeX
@misc{dc0627fad4fe01,
  title = {From Overloaded to Guaranteed: High-Throughput Multi-SLO Enforcement for LoRA-Assisted On-Premise LLM Deployment — arXiv:2610.04956},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.04956},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0627fad4fe01; sha256 e55321a577e023355f042925bc5ff511d229554f76647fed55ad8120ef4819d2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.04956. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As Large Language Models (LLMs) become essential in privacy-sensitive sectors like hospitals and government agencies, the on-premise LLM servers offer a cost-effective and secure alternative to public cloud services. However, these resource-constrained servers struggle to guarantee heterogeneous Service Level Objectives (SLOs) when serving multiple LoRA-adapted services simultaneously. Existing serving frameworks suffer from severe SLO violations due to the computational overhead of LoRA layers and the rigid nature of batch scheduling. To address this, we propose HALO, a scheduling method tailored for LoRA-assisted on-premise LLM deployment. HALO introduces two key innovations: a spatial multiplexing strategy that overlaps Base and LoRA computations by partitioning GPU Streaming Multiprocessors (SMs), and an SLO-aware scheduler that decouples request execution based on "request-level slack." By prioritizing urgent tasks and utilizing idle budget for traffic shaping, HALO significantly mitigates resource contention. Our evaluation demonstrates that HALO minimizes SLO violations while improving throughput compared to state-of-the-art baselines.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2610.04956 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:24:20.287Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).