Metraipolis › archive › dc_43d3ead45a9c
From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systems — arXiv:2608.15127
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.15127
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T18:00:28.799Z
- SHA-256 of the archived text
2faebcccbbef2370ec2520c48a7968b0da3e7438c15c5bf77d4c4df9e8b29fb0- Stable URL of this page
https://metraipolis.com/documento/dc_43d3ead45a9c
How to cite
From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systems — arXiv:2608.15127 — arXiv (en), https://arxiv.org/abs/2608.15127 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_43d3ead45a9c (Metraipolis archive dc_43d3ead45a9c, sha256 2faebcccbbef2370ec2520c48a7968b0da3e7438c15c5bf77d4c4df9e8b29fb0)
BibTeX
@misc{dc43d3ead45a9c,
title = {From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systems — arXiv:2608.15127},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.15127},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_43d3ead45a9c; sha256 2faebcccbbef2370ec2520c48a7968b0da3e7438c15c5bf77d4c4df9e8b29fb0},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.15127. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Agentic applications are shifting AI serving from isolated model inference to long-running workloads in which LLMs coordinate tools, environments, and persistent state. However, the system behavior of these workloads---where latency, cost, and bottlenecks arise---remains poorly characterized, leaving serving systems to rely on assumptions built for conventional inference. We present AgentSysBench, a benchmark suite and measurement toolkit with ten representative agentic applications and unified systems-level instrumentation. Across controlled deployments and production traces, we identify six properties that distinguish agentic workloads from conventional LLM serving: (1) execution is heavyweight and stateful, with non-LLM components dominating latency in 5 of 10 applications and sandbox working-set memory peaking at 28 GB per session; (2) applications compose components with heterogeneous resource affinity---GPU-bound inference, memory-bound retrieval, CPU-bound sandboxes---whose task latencies diverge by up to 32x; (3) bottlenecks shift across requests, models, and deployments; (4) production sessions hold state idle for minutes to hours between active steps; (5) a control-plane tax---auxiliary LLM calls and context overhead from tool schemas and observations---crowds out productive compute and context; and (6) production traces from three applications reveal heavy cross-request redundancy in search queries and web fetches, exposing a large caching opportunity. Four design explorations demonstrate that these findings are actionable: task-aware serving reduces latency by 29--40%, communication-aware placement by up to 4.5x, state offloading reduces memory usage by 4.6x, and tool-result caching removes 35.2% of redundant search calls and saves 19.3% of aggregate search latency. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI). Categorie arXiv: Operating Systems (cs.OS) ; Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC); Multiagent Systems (cs.MA). Pagina del lavoro: https://arxiv.org/abs/2608.15127 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T16:29:13.987Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).