Metraipolis › archive › dc_bee16281db76

End-to-End Latency-Minimizing and Load-Balanced Request Scheduling for Edge LLM Inference in Agentic AI Services — arXiv:2609.17193

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.17193
License (as archived): CC0-1.0
Archived: 2026-10-15T18:32:24.235Z
SHA-256 of the archived text: 0ea3e6e95026d167fb9224057643f2397612646d4429bdab4cf2cb300f90ceb0
Stable URL of this page: https://metraipolis.com/documento/dc_bee16281db76

How to cite

End-to-End Latency-Minimizing and Load-Balanced Request Scheduling for Edge LLM Inference in Agentic AI Services — arXiv:2609.17193 — arXiv (en), https://arxiv.org/abs/2609.17193 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bee16281db76 (Metraipolis archive dc_bee16281db76, sha256 0ea3e6e95026d167fb9224057643f2397612646d4429bdab4cf2cb300f90ceb0)

BibTeX

@misc{dcbee16281db76,
  title = {End-to-End Latency-Minimizing and Load-Balanced Request Scheduling for Edge LLM Inference in Agentic AI Services — arXiv:2609.17193},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.17193},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bee16281db76; sha256 0ea3e6e95026d167fb9224057643f2397612646d4429bdab4cf2cb300f90ceb0},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17193. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language model (LLM)-powered agentic AI services increasingly demand low-latency inference, motivating the deployment of LLMs across distributed edge servers. However, heterogeneous communication and computing capabilities, together with dynamically evolving inference states, make the edge server selection for each incoming request time-varying and tightly coupled across slots. In this paper, we investigate an online request scheduling framework for edge LLM inference that jointly minimizes long-term average end-to-end latency and regulates workload distribution across heterogeneous edge servers. Two main challenges arise in this context. First, conventional latency models cannot accurately capture the fine-grained dynamics of multi-stage LLM execution. Second, the latency consequence of a scheduling decision is observed only after request completion, making immediate decision evaluation difficult. To address these challenges, we develop a cross-slot inference model that captures transmission, prefill, iteration-level decoding, and key-value (KV) cache evolution for each diverse request, and characterize server workload through a KV cache memory-time consumption metric. We propose the LYREO approach that transforms the long-term load-balancing constraint via Lyapunov optimization and employs reward redistribution with sequencebased return prediction to convert delayed outcomes into timely learning signals for earlier decisions. Simulations under various configurations demonstrate that LYREO consistently achieves lower latency and more balanced load distribution than representative learning-based and heuristic baseline schemes.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-16 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.17193 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T18:03:09.521Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-16?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).