Metraipolis › archive › dc_a0eb95690629

Apto: Self-Adaptive CPU-GPU Hybrid MoE Inference on Shared PCs — arXiv:2611.01859

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.01859
License (as archived)
CC0-1.0
Archived
2026-11-03T11:12:39.675Z
SHA-256 of the archived text
a47ab6e35ff3818fae48cd3e3693e65e0f82f76c7a0c49c3cd54719631dadf11
Stable URL of this page
https://metraipolis.com/documento/dc_a0eb95690629

How to cite

Apto: Self-Adaptive CPU-GPU Hybrid MoE Inference on Shared PCs — arXiv:2611.01859 — arXiv (en), https://arxiv.org/abs/2611.01859 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a0eb95690629 (Metraipolis archive dc_a0eb95690629, sha256 a47ab6e35ff3818fae48cd3e3693e65e0f82f76c7a0c49c3cd54719631dadf11)
BibTeX
@misc{dca0eb95690629,
  title = {Apto: Self-Adaptive CPU-GPU Hybrid MoE Inference on Shared PCs — arXiv:2611.01859},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01859},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a0eb95690629; sha256 a47ab6e35ff3818fae48cd3e3693e65e0f82f76c7a0c49c3cd54719631dadf11},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01859. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

CPU/GPU hybrid inference enables Mixture-of-Experts (MoE) models to run on memory-constrained PCs by keeping all expert weights in host memory and executing activated experts across both processors. Existing hybrid systems, however, assume exclusive resource access and determine an upfront CPU/GPU partition, so CPU contention from colocated applications can severely imbalance the two execution paths. This paper presents Apto, a self-adaptive hybrid inference system that applies dynamic scheduling to expert execution. The CPU and GPU executors claim experts from opposite ends of a dual-ended queue as they progress, allowing the CPU/GPU work split to adapt to their execution progress without predicting contention. Apto further uses affinity-aware scheduling to expose each executor's preferred experts at its queue end and cancellable tail racing to mitigate final-expert stragglers. Compared with state-of-the-art GPU-only and hybrid baselines, Apto achieves average prefill and decode speedups of 2.96$\times$ and 1.52$\times$ under standalone execution, and 5.97$\times$ and 3.45$\times$ under concurrent CPU workloads. It also accelerates real-world coding-agent applications by 2.02$\times$ on average.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2611.01859 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T11:00:07.152Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).