Metraipolis › archive › dc_a9c1fcc482fb

PDScope: Breaking the Prefill-Decode Seesaw for Memory-Bound MoE Inference — arXiv:2610.39216

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.39216
License (as archived)
CC0-1.0
Archived
2026-10-31T06:19:32.331Z
SHA-256 of the archived text
41b0cc43ffb481ac2e5058eedadbb9a0d2f93f6caa55e5ec0ce6eec06170edce
Stable URL of this page
https://metraipolis.com/documento/dc_a9c1fcc482fb

How to cite

PDScope: Breaking the Prefill-Decode Seesaw for Memory-Bound MoE Inference — arXiv:2610.39216 — arXiv (en), https://arxiv.org/abs/2610.39216 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a9c1fcc482fb (Metraipolis archive dc_a9c1fcc482fb, sha256 41b0cc43ffb481ac2e5058eedadbb9a0d2f93f6caa55e5ec0ce6eec06170edce)
BibTeX
@misc{dca9c1fcc482fb,
  title = {PDScope: Breaking the Prefill-Decode Seesaw for Memory-Bound MoE Inference — arXiv:2610.39216},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.39216},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a9c1fcc482fb; sha256 41b0cc43ffb481ac2e5058eedadbb9a0d2f93f6caa55e5ec0ce6eec06170edce},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.39216. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Local deployment of MoE-based LLMs on CPU-GPU nodes is attractive for cost and privacy reasons, yet limited GPU memory forces most experts into CPU memory. Prefill is then compute-bound and benefits from aggressive GPU loading, while decode is I/O-bound and requires conservative PCIe transfers. These demands cannot be amortized under low batching---typical of local assistants and document tools---forcing phase-agnostic schedulers into a seesaw: favoring prefill's aggressive loading injects unmaskable I/O that stalls decode, while favoring decode's conservative transfers starves prefill's GPU of work. We observe that expert routing is predictable in phase-specific ways, allowing us to decouple the two phases and optimize prediction, scheduling, and execution independently for each phase. PDScope predicts expert activations to guide placement and prefetching, adapts these policies to each phase's bottleneck via cross-layer cost modeling, and compiles decisions into overhead-free pipelines using a static computation graph, cache prefetching, and zero-copy asynchronous I/O. On mainstream MoE models, PDScope achieves 2.5-10.3x higher prefill throughput and 2.1-6.1x higher decode throughput than existing systems, showing that phase-aware co-design unlocks efficient MoE inference under memory constraints and extends naturally to multi-batch settings.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2610.39216 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T06:07:53.377Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).