Metraipolis › archive › dc_5eef0141816d

BenchmarkAnything: Agent-Driven Construction of Simulator-Ready Microarchitecture Benchmarks — arXiv:2610.09447

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.09447
License (as archived)
CC0-1.0
Archived
2026-10-08T11:29:30.491Z
SHA-256 of the archived text
27ee1ebea0d9546f2af95fa56ba8091260add3579df8433bb69a5ba46283eef7
Stable URL of this page
https://metraipolis.com/documento/dc_5eef0141816d

How to cite

BenchmarkAnything: Agent-Driven Construction of Simulator-Ready Microarchitecture Benchmarks — arXiv:2610.09447 — arXiv (en), https://arxiv.org/abs/2610.09447 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5eef0141816d (Metraipolis archive dc_5eef0141816d, sha256 27ee1ebea0d9546f2af95fa56ba8091260add3579df8433bb69a5ba46283eef7)
BibTeX
@misc{dc5eef0141816d,
  title = {BenchmarkAnything: Agent-Driven Construction of Simulator-Ready Microarchitecture Benchmarks — arXiv:2610.09447},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.09447},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5eef0141816d; sha256 27ee1ebea0d9546f2af95fa56ba8091260add3579df8433bb69a5ba46283eef7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.09447. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

The selection of benchmark workloads is of paramount importance in computer architecture, as it establishes the yardstick against which architectural innovations are measured and guided. Yet for decades, the SPEC benchmark suites, comprising merely tens of workloads, have been the de facto standard in academic architectural research, where they are frequently treated as a principal evaluation and optimization target. When a suite this small is relied upon so heavily, it risks architectural overfitting; as our research and prior studies demonstrate, an overly narrow focus can mislead design decisions by overvaluing certain innovations, producing cores that excel on SPEC benchmarks yet underperform on broader, realistic workloads. To mitigate this overfitting, adopting a large, comprehensive benchmark suite is the natural solution. However, the immense engineering effort required to strip software into the clean, interference-free binary executables demanded by simulators often makes this highly impractical. In this work, we demonstrate that AI agents provide an elegant solution to this challenge. Rather than manually curating yet another static benchmark suite, we introduce an agent-driven workflow capable of autonomously transforming arbitrary open-source repositories into simulator-ready executables. This automated approach makes workload collection highly scalable, allowing us to rapidly harvest hundreds of diverse applications from public repositories into our benchmark suite. Through a comparative analysis of our agent-generated suite against SPEC, we show that it not only achieves higher-fidelity performance assessments but also uncovers novel architectural insights that traditional, static suites fail to expose.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.AR, nuovi lavori del giorno).
Categorie arXiv: Hardware Architecture (cs.AR).
Pagina del lavoro: https://arxiv.org/abs/2610.09447 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T10:29:21.572Z dalla pagina https://arxiv.org/list/cs.AR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).