Metraipolis › archive › dc_489f44cc36bd

AgentReplay: Token-Wise Trace Replay Is Essential for Fair Serving System Performance Benchmarking — arXiv:2609.32283

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.32283
License (as archived)
CC0-1.0
Archived
2026-10-07T22:12:01.237Z
SHA-256 of the archived text
cd1248076665dfb26d7e2c7a6cc0949258eb28ff905155cfe7caa2dbd3f4234e
Stable URL of this page
https://metraipolis.com/documento/dc_489f44cc36bd

How to cite

AgentReplay: Token-Wise Trace Replay Is Essential for Fair Serving System Performance Benchmarking — arXiv:2609.32283 — arXiv (en), https://arxiv.org/abs/2609.32283 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_489f44cc36bd (Metraipolis archive dc_489f44cc36bd, sha256 cd1248076665dfb26d7e2c7a6cc0949258eb28ff905155cfe7caa2dbd3f4234e)
BibTeX
@misc{dc489f44cc36bd,
  title = {AgentReplay: Token-Wise Trace Replay Is Essential for Fair Serving System Performance Benchmarking — arXiv:2609.32283},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32283},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_489f44cc36bd; sha256 cd1248076665dfb26d7e2c7a6cc0949258eb28ff905155cfe7caa2dbd3f4234e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32283. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

LLM-based agents execute multi-turn workflows with interleaved model inference and tool calls, making efficient serving increasingly important. However, evaluating serving optimizations is challenging because identical tasks can produce different execution trajectories. Changes in generated tokens can alter subsequent prompts, tool calls, and reasoning turns, making it difficult to distinguish system improvements from workload variation. Greedy decoding does not guarantee identical outputs, while replaying only sequence lengths loses token information that affects prefix caching and mixture-of-experts (MoE) routing. To address these problems, we propose AgentReplay, a configurable trace record-and-replay framework for agent serving. AgentReplay records input/output tokens, expert selections, request dependencies, and tool durations. During replay, it forces the recorded output tokens while performing normal autoregressive computation, with optional controls for MoE expert selection and tool delays. This allows different systems to execute the same recorded workload while retaining their own batching, scheduling, and parallelization decisions. We further separate trajectory generation from performance evaluation, enabling compatible smaller models to replay long-horizon traces collected with more capable models. Our experiments show that token-wise replay in AgentReplay effectively eliminates workload variation that greedy decoding and length-wise replay cannot avoid, enabling fairer performance comparisons across serving configurations.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-29 in cs.DC).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2609.32283 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T21:53:13.651Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).