Metraipolis › archive › dc_a033cd0b2eb0

PhaseGate: Phase-Aware CPU Retrieval Scheduling for On-Device LLMs on Unified Memory — arXiv:2610.04537

Source: arXiv
Original: https://arxiv.org/abs/2610.04537
License (as archived): CC0-1.0
Archived: 2026-10-06T05:29:06.899Z
SHA-256 of the archived text: e7bbc3f090fc223ec394911eb1d0e4261d2c8a3109b46339de3f50394a5b2bcb
Stable URL of this page: https://metraipolis.com/documento/dc_a033cd0b2eb0

How to cite

PhaseGate: Phase-Aware CPU Retrieval Scheduling for On-Device LLMs on Unified Memory — arXiv:2610.04537 — arXiv, https://arxiv.org/abs/2610.04537 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a033cd0b2eb0 (Metraipolis archive dc_a033cd0b2eb0, sha256 e7bbc3f090fc223ec394911eb1d0e4261d2c8a3109b46339de3f50394a5b2bcb)

BibTeX

@misc{dca033cd0b2eb0,
  title = {PhaseGate: Phase-Aware CPU Retrieval Scheduling for On-Device LLMs on Unified Memory — arXiv:2610.04537},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.04537},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a033cd0b2eb0; sha256 e7bbc3f090fc223ec394911eb1d0e4261d2c8a3109b46339de3f50394a5b2bcb},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.04537. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

On-device assistants run GPU-based LLM inference alongside CPU retrieval on unified-memory systems. Under a saturated local-retrieval workload, four concurrent retrieval workers raise 95th-percentile (p95) decode latency by 60-61% on two M4 systems, whereas prefill latency rises by only 5.7-6.9%. We study LLM phase as an admission signal for independent CPU retrieval under controlled LLM workloads. PHASEGATE calibrates separate concurrency limits for prefill and decode, selecting four and one on our base-M4 configuration. Under a backlogged queue, it achieves 2.0 times the aggregate retrieval throughput of the best tested feasible fixed policy, with both p95 LLM latency metrics within 1.25 times their no-retrieval baselines in all seven held-out runs. A phase-blind control, TimeGate, uses the same two limits on a calibration-derived schedule without observing LLM phase. It achieves similar retrieval throughput but violates the output-token latency limit in every run. M2 and M2 Pro Mac minis reproduce the policy ordering, while output-length sweeps show that the advantage narrows as decode occupies more of each request.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Distributed, Parallel, and Cluster Computing (cs.DC); Performance (cs.PF).
Pagina del lavoro: https://arxiv.org/abs/2610.04537 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:24:35.331Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).