Metraipolis › archive › dc_a0eb95690629
Apto: Self-Adaptive CPU-GPU Hybrid MoE Inference on Shared PCs — arXiv:2611.01859
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.01859
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T11:12:39.675Z
- SHA-256 of the archived text
a47ab6e35ff3818fae48cd3e3693e65e0f82f76c7a0c49c3cd54719631dadf11- Stable URL of this page
https://metraipolis.com/documento/dc_a0eb95690629
How to cite
Apto: Self-Adaptive CPU-GPU Hybrid MoE Inference on Shared PCs — arXiv:2611.01859 — arXiv (en), https://arxiv.org/abs/2611.01859 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a0eb95690629 (Metraipolis archive dc_a0eb95690629, sha256 a47ab6e35ff3818fae48cd3e3693e65e0f82f76c7a0c49c3cd54719631dadf11)
BibTeX
@misc{dca0eb95690629,
title = {Apto: Self-Adaptive CPU-GPU Hybrid MoE Inference on Shared PCs — arXiv:2611.01859},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.01859},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a0eb95690629; sha256 a47ab6e35ff3818fae48cd3e3693e65e0f82f76c7a0c49c3cd54719631dadf11},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01859. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
CPU/GPU hybrid inference enables Mixture-of-Experts (MoE) models to run on memory-constrained PCs by keeping all expert weights in host memory and executing activated experts across both processors. Existing hybrid systems, however, assume exclusive resource access and determine an upfront CPU/GPU partition, so CPU contention from colocated applications can severely imbalance the two execution paths. This paper presents Apto, a self-adaptive hybrid inference system that applies dynamic scheduling to expert execution. The CPU and GPU executors claim experts from opposite ends of a dual-ended queue as they progress, allowing the CPU/GPU work split to adapt to their execution progress without predicting contention. Apto further uses affinity-aware scheduling to expose each executor's preferred experts at its queue end and cancellable tail racing to mitigate final-expert stragglers. Compared with state-of-the-art GPU-only and hybrid baselines, Apto achieves average prefill and decode speedups of 2.96$\times$ and 1.52$\times$ under standalone execution, and 5.97$\times$ and 3.45$\times$ under concurrent CPU workloads. It also accelerates real-world coding-agent applications by 2.02$\times$ on average. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC). Pagina del lavoro: https://arxiv.org/abs/2611.01859 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T11:00:07.152Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).