Metraipolis › archive › dc_3323a7cecdbb

EdgeAgent: Orchestrating On-Device LLM inference for End-User Multi-Agent Systems on CPU-GPU Unified Memory Architectures — arXiv:2610.03394

Source
arXiv
Original
https://arxiv.org/abs/2610.03394
License (as archived)
CC0-1.0
Archived
2026-10-05T22:48:07.933Z
SHA-256 of the archived text
e0009fdb87bbc8bd544a0b8b1e17d5ac211f511286b27a7fc3309db16bae369a
Stable URL of this page
https://metraipolis.com/documento/dc_3323a7cecdbb

How to cite

EdgeAgent: Orchestrating On-Device LLM inference for End-User Multi-Agent Systems on CPU-GPU Unified Memory Architectures — arXiv:2610.03394 — arXiv, https://arxiv.org/abs/2610.03394 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3323a7cecdbb (Metraipolis archive dc_3323a7cecdbb, sha256 e0009fdb87bbc8bd544a0b8b1e17d5ac211f511286b27a7fc3309db16bae369a)
BibTeX
@misc{dc3323a7cecdbb,
  title = {EdgeAgent: Orchestrating On-Device LLM inference for End-User Multi-Agent Systems on CPU-GPU Unified Memory Architectures — arXiv:2610.03394},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.03394},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3323a7cecdbb; sha256 e0009fdb87bbc8bd544a0b8b1e17d5ac211f511286b27a7fc3309db16bae369a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.03394. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Emerging multi-agent LLMs demand privacy-preserving edge deployment, yet current inference systems struggle with these collaborative workflows. Specifically, the memory-bound decode phase causes severe bus contention on unified memory architectures (UMA), paralyzing naive CPU-GPU co-execution. Furthermore, speculative decoding in multi-agent workloads faces extreme variance in drafting difficulty, alternating between complex reasoning and predictable structured generation. Compounded by frequent tool-induced stalls, this highly fragmented execution severely underutilizes hardware and defeats traditional static batching. We present EdgeAgent, a cross-layer inference system explicitly co-designed for edge UMA and multi-agent workloads. At the micro-architectural level, it bypasses rigid graph-compiler constraints to enable zero-copy UMA-aware tensor parallelism, utilizing asymmetric memory layouts to fully saturate both CPU and GPU compute units. At the scheduling level, it dynamically allocates draft budgets based on real-time sequence predictability to bound bandwidth waste. Concurrently, an asynchronous suspend-and-yield mechanism actively evicts stalled agents, ensuring continuous hardware saturation during unpredictable tool invocations. Extensive evaluations on an Apple M4 SoC demonstrate that the UMA-aware execution alone contributes a 1.29x speedup over batched speculative decoding. Adding the agent-aware scheduling lifts the full EdgeAgent system to a 1.77x speedup under extreme tool-use latencies.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.MA, nuovi lavori del giorno).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Multiagent Systems (cs.MA).
Pagina del lavoro: https://arxiv.org/abs/2610.03394 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-05T22:45:40.658Z dalla pagina https://arxiv.org/list/cs.MA/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).