Metraipolis › archive › dc_7a7fa4b54cf7

LoRAForge: Co-Designing Post-Hoc Compression and Execution for Efficient Multi-LoRA Serving — arXiv:2611.01256

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.01256
License (as archived)
CC0-1.0
Archived
2026-11-03T10:35:46.913Z
SHA-256 of the archived text
4f2cadb5231fb19204d3e454f7472453ed6a6c840181a1ee60226ec33e63502b
Stable URL of this page
https://metraipolis.com/documento/dc_7a7fa4b54cf7

How to cite

LoRAForge: Co-Designing Post-Hoc Compression and Execution for Efficient Multi-LoRA Serving — arXiv:2611.01256 — arXiv (en), https://arxiv.org/abs/2611.01256 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7a7fa4b54cf7 (Metraipolis archive dc_7a7fa4b54cf7, sha256 4f2cadb5231fb19204d3e454f7472453ed6a6c840181a1ee60226ec33e63502b)
BibTeX
@misc{dc7a7fa4b54cf7,
  title = {LoRAForge: Co-Designing Post-Hoc Compression and Execution for Efficient Multi-LoRA Serving — arXiv:2611.01256},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01256},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7a7fa4b54cf7; sha256 4f2cadb5231fb19204d3e454f7472453ed6a6c840181a1ee60226ec33e63502b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01256. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multi-LoRA serving allows many task-specific adapters to share a common base model, but scaling to large adapter fleets introduces substantial storage, loading, and execution overheads. Post-hoc compression can reduce the memory footprint of trained LoRA adapters, yet smaller ranks do not automatically translate into faster serving. For instance, padding, factor access, and generic kernels can obscure the retained low-rank structure, while independently recompressing thousands of modules also incurs significant preparation overhead. We present LoRAForge, a system connecting batched factor construction to rank-specialized execution through an explicit retained-rank representation. It supports two allocation policies: uniform retention ratios (LoRAForge-U) and spectral-energy budget allocation (LoRAForge-S). Both share FlashTSQR factor construction and the same serving backend. FlashTSQR exploits parallelism across modules and row tiles, accelerating factor construction by up to $2.28\times$ over matched library QR. In a fixed-work Qwen control, the shared runtime adds $1.17$--$1.22\times$ throughput across both compressed policies, with negligible gains on original adapters. Across 85 trained open-weighted Qwen and Llama adapters, the spectral policy's floored quality configuration retains about 7.7% of adapter parameters with fleet-average score drops of at most 1.2 points on a 0--100 scale. The same rank-floored $1/12$ spectral exports achieve substantial token-throughput gains on both AMD and NVIDIA GPUs.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.01256 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:29:48.755Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).