Metraipolis › archive › dc_a9bafffcc3f8
MAPS: Memory-Aware Predictive Scheduling Framework for Large Language Model Serving — arXiv:2609.15359
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.15359
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T22:47:57.940Z
- SHA-256 of the archived text
f916b67a0523c19b5496af3a89c2189e3ea24f34b5c643df20ceee0cd0c07360- Stable URL of this page
https://metraipolis.com/documento/dc_a9bafffcc3f8
How to cite
MAPS: Memory-Aware Predictive Scheduling Framework for Large Language Model Serving — arXiv:2609.15359 — arXiv (en), https://arxiv.org/abs/2609.15359 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a9bafffcc3f8 (Metraipolis archive dc_a9bafffcc3f8, sha256 f916b67a0523c19b5496af3a89c2189e3ea24f34b5c643df20ceee0cd0c07360)
BibTeX
@misc{dca9bafffcc3f8,
title = {MAPS: Memory-Aware Predictive Scheduling Framework for Large Language Model Serving — arXiv:2609.15359},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.15359},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a9bafffcc3f8; sha256 f916b67a0523c19b5496af3a89c2189e3ea24f34b5c643df20ceee0cd0c07360},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.15359. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
The surge of large language model (LLM) applications on personal devices imposes massive, bursty workloads on cloud serving infrastructure. While prefill-decode disaggregation improves throughput and scalability, memory-bound decode instances often suffer from persistent load imbalance, as output lengths are unknown when requests arrive at the cloud. To address this, we propose MAPS, a Memory-Aware Predictive Scheduling framework tailored for disaggregated LLM serving. MAPS performs device-assisted speculative output length prediction overlapped with cloud-side prefilling, incurring negligible latency overhead. To handle generation uncertainty, MAPS applies uncertainty-aware calibration to derive output-length upper bounds with target coverage, enabling safe scheduling decisions. Building on these bounds, MAPS employs a hierarchical global-local scheduling strategy to mitigate inter-decoder queue buildup and intra-decoder head-of-line blocking. Extensive experiments on two real-world workloads and two LLMs show that MAPS significantly outperforms three state-of-the-art systems, reducing average end-to-end latency by 42.6 and tail latency by up to 84.8. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.15359 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T22:29:03.306Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).