Metraipolis › archive › dc_3323a7cecdbb
EdgeAgent: Orchestrating On-Device LLM inference for End-User Multi-Agent Systems on CPU-GPU Unified Memory Architectures — arXiv:2610.03394
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.03394
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-05T22:48:07.933Z
- SHA-256 of the archived text
e0009fdb87bbc8bd544a0b8b1e17d5ac211f511286b27a7fc3309db16bae369a- Stable URL of this page
https://metraipolis.com/documento/dc_3323a7cecdbb
How to cite
EdgeAgent: Orchestrating On-Device LLM inference for End-User Multi-Agent Systems on CPU-GPU Unified Memory Architectures — arXiv:2610.03394 — arXiv, https://arxiv.org/abs/2610.03394 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3323a7cecdbb (Metraipolis archive dc_3323a7cecdbb, sha256 e0009fdb87bbc8bd544a0b8b1e17d5ac211f511286b27a7fc3309db16bae369a)
BibTeX
@misc{dc3323a7cecdbb,
title = {EdgeAgent: Orchestrating On-Device LLM inference for End-User Multi-Agent Systems on CPU-GPU Unified Memory Architectures — arXiv:2610.03394},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.03394},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3323a7cecdbb; sha256 e0009fdb87bbc8bd544a0b8b1e17d5ac211f511286b27a7fc3309db16bae369a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.03394. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Emerging multi-agent LLMs demand privacy-preserving edge deployment, yet current inference systems struggle with these collaborative workflows. Specifically, the memory-bound decode phase causes severe bus contention on unified memory architectures (UMA), paralyzing naive CPU-GPU co-execution. Furthermore, speculative decoding in multi-agent workloads faces extreme variance in drafting difficulty, alternating between complex reasoning and predictable structured generation. Compounded by frequent tool-induced stalls, this highly fragmented execution severely underutilizes hardware and defeats traditional static batching. We present EdgeAgent, a cross-layer inference system explicitly co-designed for edge UMA and multi-agent workloads. At the micro-architectural level, it bypasses rigid graph-compiler constraints to enable zero-copy UMA-aware tensor parallelism, utilizing asymmetric memory layouts to fully saturate both CPU and GPU compute units. At the scheduling level, it dynamically allocates draft budgets based on real-time sequence predictability to bound bandwidth waste. Concurrently, an asynchronous suspend-and-yield mechanism actively evicts stalled agents, ensuring continuous hardware saturation during unpredictable tool invocations. Extensive evaluations on an Apple M4 SoC demonstrate that the UMA-aware execution alone contributes a 1.29x speedup over batched speculative decoding. Adding the agent-aware scheduling lifts the full EdgeAgent system to a 1.77x speedup under extreme tool-use latencies. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.MA, nuovi lavori del giorno). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Multiagent Systems (cs.MA). Pagina del lavoro: https://arxiv.org/abs/2610.03394 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-05T22:45:40.658Z dalla pagina https://arxiv.org/list/cs.MA/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).