Metraipolis › archive › dc_032cd7e51eb1

BTS-AgentBench: A Deterministic, Replayable Pipeline from Read-Only Telemetry Logs to Agent Benchmarks — arXiv:2608.27334

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.27334
License (as archived)
CC0-1.0
Archived
2026-10-17T15:20:30.075Z
SHA-256 of the archived text
bc101ade4152d1a7ab2748e091756adb5add7d11f7bf855818dc0bfa981d05dd
Stable URL of this page
https://metraipolis.com/documento/dc_032cd7e51eb1

How to cite

BTS-AgentBench: A Deterministic, Replayable Pipeline from Read-Only Telemetry Logs to Agent Benchmarks — arXiv:2608.27334 — arXiv (en), https://arxiv.org/abs/2608.27334 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_032cd7e51eb1 (Metraipolis archive dc_032cd7e51eb1, sha256 bc101ade4152d1a7ab2748e091756adb5add7d11f7bf855818dc0bfa981d05dd)
BibTeX
@misc{dc032cd7e51eb1,
  title = {BTS-AgentBench: A Deterministic, Replayable Pipeline from Read-Only Telemetry Logs to Agent Benchmarks — arXiv:2608.27334},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.27334},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_032cd7e51eb1; sha256 bc101ade4152d1a7ab2748e091756adb5add7d11f7bf855818dc0bfa981d05dd},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.27334. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Operational facilities accumulate extensive local telemetry, but agent-ready interactive evaluations with source-verifiable answers are costly to construct from such archives. Site-specific measurement histories also motivate a local conversion workflow that can operate where raw operational data must remain under organizational control. We introduce BTS-AgentBench, a deterministic compiler that transforms read-only telemetry and equipment metadata into tool-backed, multi-turn agent tasks, phase-level targets, evidence and executable evaluation contracts. The BTS release includes 532 episodes across nine families; a second-corpus port produces 204 XAI4HEAT episodes. Frozen-input replay reproduces released artifacts byte-for-byte, and independent raw-source checks cover all 89 BTS and 41 XAI4HEAT test episodes. Replayable provenance links 1,170 trial identities to recorded execution and scorer conditions, while 1,125 completed traces retain model responses and tool results. An obligation-level comparison of 216 scored failures found that linked records exposed additional recorded failed obligations in 202 cases (93.5\%), including tool arguments, stream identity, state carryover and missing interaction steps. The pipeline thereby supports source-grounded task generation, exact reconstruction and auditable diagnosis of agent executions. Code, released benchmarks, and reproduction instructions are available at this https URL

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-28 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2608.27334 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T15:15:18.093Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).