Metraipolis › archive › dc_df9af514abdf
EAServe: Encode-Aware Disaggregated Serving for Multimodal Large Language Models — arXiv:2609.31551
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.31551
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T02:04:54.144Z
- SHA-256 of the archived text
3c7fb25ebfae4e82b0fa60fc11bd3a920fbc24536be05a6958b84234d1917f21- Stable URL of this page
https://metraipolis.com/documento/dc_df9af514abdf
How to cite
EAServe: Encode-Aware Disaggregated Serving for Multimodal Large Language Models — arXiv:2609.31551 — arXiv (en), https://arxiv.org/abs/2609.31551 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_df9af514abdf (Metraipolis archive dc_df9af514abdf, sha256 3c7fb25ebfae4e82b0fa60fc11bd3a920fbc24536be05a6958b84234d1917f21)
BibTeX
@misc{dcdf9af514abdf,
title = {EAServe: Encode-Aware Disaggregated Serving for Multimodal Large Language Models — arXiv:2609.31551},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.31551},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_df9af514abdf; sha256 3c7fb25ebfae4e82b0fa60fc11bd3a920fbc24536be05a6958b84234d1917f21},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31551. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Disaggregating the two stages, Prefill and Decode, onto separate GPU pools is now a standard optimization for (text-only) LLM serving. However, multimodal LLMs (MLLMs), which add a third phase, Encode, pose new challenges for resource allocation. Encode turns images, video, or audio into embeddings that the language model can consume, yielding a three-stage Encode-Prefill-Decode (EPD) pipeline. Existing frameworks offer only partial answers: text-only PD systems lack Encode, while EPD frameworks expose it as a separate service without regulating downstream request flow. The pipeline also carries a structural resource imbalance: every request enters through Encode before downstream work can begin, yet per-request execution leaves the encode GPU severely underutilized even at high loads, starving the downstream Prefill and Decode workers. Addressing this, we reposition Encode as the control point of the EPD pipeline, exposing three tightly coupled dimensions: when work enters downstream, where prefill executes, and how the GPU is shared. We instantiate this in EAServe across two co-designed layers. Its runtime manages load-adaptive micro-batching, rate-controlled partial offload to a co-resident prefill worker, and dynamic SM partitioning for predictable co-location. The configuration layer, Hybrid Auto Selection (HAS), navigates the joint space of GPU allocation, encode batch size, and offload ratio by pruning unbalanced allocations with per-stage capacity profiling and refining the remainder through TPE-based Bayesian optimization. Evaluated on three MLLM architectures spanning image, video, and audio, EAServe delivers up to 4.3x and 1.7x higher goodput than NVIDIA Dynamo and vLLM, respectively, under identical SLO constraints, sustains more balanced and higher GPU utilization across the EPD pipeline, and reaches near-optimal configurations faster than baseline search methods. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-28 in cs.DC). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Machine Learning (cs.LG); Performance (cs.PF). Pagina del lavoro: https://arxiv.org/abs/2609.31551 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T01:40:22.294Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).