Metraipolis › archive › dc_b4f84128fe54
DPS: Dual-Mode Precision LLM Serving with Semi-Unified Memory — arXiv:2609.34380
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.34380
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T21:02:30.675Z
- SHA-256 of the archived text
97517eedf51461fa965ba2a21381f527cb2f3d7ada47aedf8f471fbc239d696f- Stable URL of this page
https://metraipolis.com/documento/dc_b4f84128fe54
How to cite
DPS: Dual-Mode Precision LLM Serving with Semi-Unified Memory — arXiv:2609.34380 — arXiv (en), https://arxiv.org/abs/2609.34380 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b4f84128fe54 (Metraipolis archive dc_b4f84128fe54, sha256 97517eedf51461fa965ba2a21381f527cb2f3d7ada47aedf8f471fbc239d696f)
BibTeX
@misc{dcb4f84128fe54,
title = {DPS: Dual-Mode Precision LLM Serving with Semi-Unified Memory — arXiv:2609.34380},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.34380},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b4f84128fe54; sha256 97517eedf51461fa965ba2a21381f527cb2f3d7ada47aedf8f471fbc239d696f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34380. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Existing LLM serving systems virtualize and optimize KV-cache memory, but treat model-weight memory as fixed throughout execution. Recent work on multi-precision model representations challenges this design by allowing a single stored model to support both full-accuracy and lower-precision execution, making the effective weight footprint runtime-dependent. This creates an opportunity under bursty workloads, where temporary spikes in KV-cache demand often determine throughput and SLO compliance. We present DPS, a dual-precision LLM serving system that turns weight memory into an elastic resource: under normal load, DPS serves the full-accuracy model; under KV pressure, it switches to a nested, lower-precision variant and repurposes unused weight memory for KV cache blocks. DPS is built on Semi-Unified Memory (SUM), which partitions the weight region into a persistent lower-precision sub-region and a shared region that alternates between residual weight tensors and KV-cache blocks, preserving compatibility with paged KV-cache management. We implement DPS on top of vLLM and evaluate it across both dense and MoE models and various production workload traces. Our results show that \sysname improves sustained throughput by $2.1$--$3.3\times$ and effective pass@1 by up to $+41$\,pp over Static FP16, while preserving FP16-class accuracy. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET); Performance (cs.PF); Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2609.34380 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:29:02.977Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).