Metraipolis › archive › dc_20de5fd08d62

Semantics, Workflows, and Infrastructure: Understanding Agent Serving at Production Scale — arXiv:2609.34432

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.34432
License (as archived)
CC0-1.0
Archived
2026-10-07T22:12:12.409Z
SHA-256 of the archived text
3a3c2f4fae1187485b7f1802cc2b29f0a7645f66b447209460a17dd4172a432b
Stable URL of this page
https://metraipolis.com/documento/dc_20de5fd08d62

How to cite

Semantics, Workflows, and Infrastructure: Understanding Agent Serving at Production Scale — arXiv:2609.34432 — arXiv (en), https://arxiv.org/abs/2609.34432 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_20de5fd08d62 (Metraipolis archive dc_20de5fd08d62, sha256 3a3c2f4fae1187485b7f1802cc2b29f0a7645f66b447209460a17dd4172a432b)
BibTeX
@misc{dc20de5fd08d62,
  title = {Semantics, Workflows, and Infrastructure: Understanding Agent Serving at Production Scale — arXiv:2609.34432},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.34432},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_20de5fd08d62; sha256 3a3c2f4fae1187485b7f1802cc2b29f0a7645f66b447209460a17dd4172a432b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34432. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language model (LLM) agents execute applications through a workflow of inference requests with tool calls and user interactions. Serving these applications at production scale requires understanding how application behavior shapes inference demand and for guiding efficient execution. Recent characterization studies provide request-level workload measurements and agent execution analysis. However, an end-to-end view connecting task initiation, workflow execution, and inference infrastructure remains unexplored. In this paper, we analyze a two-week trace of 11.7 million requests from a large-scale production platform for general-purpose agents, backed by inference infrastructure comprising over 10k GPUs. We characterize the platform at three connected levels: task-level initiation semantics, workflow-level execution patterns, and infrastructure level serving demands. Our measurements reveal workload patterns such as highly skewed request volumes across sessions, rare execution overlap among logical sibling requests, and context reuse across task boundaries. Building on these observations, we analyze deployment implications and identify open problems to guide future research on agent serving systems.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-29 in cs.DC).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2609.34432 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T21:53:13.711Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).