Metraipolis › archive › dc_6dc54823717a
Benchmark, Agent, and Tool-Use Harnesses for Large Language Models: A Taxonomy of Evaluation Infrastructure (2026) · DOI 10.20944/preprints202609.0236.v1
- Source
- Metraipolis archive
- Original
- https://doi.org/10.20944/preprints202609.0236.v1
- License (as archived)
- CC-BY-4.0
- Archived
- 2026-10-07T10:36:05.520Z
- SHA-256 of the archived text
737639a8630de544983960536b85301128bb80d74e4a3c3c34326fcc05c0b85c- Stable URL of this page
https://metraipolis.com/documento/dc_6dc54823717a
How to cite
Benchmark, Agent, and Tool-Use Harnesses for Large Language Models: A Taxonomy of Evaluation Infrastructure (2026) · DOI 10.20944/preprints202609.0236.v1 — Metraipolis archive, https://doi.org/10.20944/preprints202609.0236.v1 (CC-BY-4.0); archived copy: https://metraipolis.com/documento/dc_6dc54823717a (Metraipolis archive dc_6dc54823717a, sha256 737639a8630de544983960536b85301128bb80d74e4a3c3c34326fcc05c0b85c)
BibTeX
@misc{dc6dc54823717a,
title = {Benchmark, Agent, and Tool-Use Harnesses for Large Language Models: A Taxonomy of Evaluation Infrastructure (2026) · DOI 10.20944/preprints202609.0236.v1},
howpublished = {Metraipolis archive},
url = {https://doi.org/10.20944/preprints202609.0236.v1},
note = {License CC-BY-4.0; archived copy https://metraipolis.com/documento/dc_6dc54823717a; sha256 737639a8630de544983960536b85301128bb80d74e4a3c3c34326fcc05c0b85c},
year = {2026}
}Archived text
Reproduced under CC-BY-4.0. Attribution: Metraipolis archive, https://doi.org/10.20944/preprints202609.0236.v1. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Articolo recente con licenza aperta: scheda dai metadati Crossref (posted-content). Titolo: Benchmark, Agent, and Tool-Use Harnesses for Large Language Models: A Taxonomy of Evaluation Infrastructure Autori: Ziqian Bi, Junfeng Hao, Yue Ma, Ye Tao, Xin Liang Chia Pubblicato: 2026-09-03 · editore: MDPI AG DOI: https://doi.org/10.20944/preprints202609.0236.v1 Licenza dell'articolo: CC-BY-4.0 Abstract (testo degli autori, CC-BY-4.0): Large language model evaluation has moved from static benchmark scripts to executable harnesses:systems that bind datasets, prompts, model adapters, inference backends, tools, environments, sandboxes,scorers, traces, and leaderboards into a measurement contract. This survey explains why that layer nowdetermines what a benchmark score means. We distinguish three harness families. Benchmark harnessesmake model comparison possible by fixing task registries, prompt templates, inference adapters, answerparsers, metrics, and aggregation rules. Agent harnesses turn evaluation into an environment experimentin which repositories, browsers, operating systems, mobile apps, users, databases, reset procedures, andexecutable oracles become part of the measured system. Tool-use harnesses isolate the interface problem: retrieving tool documentation, selecting APIs, formatting arguments, deciding when not to call atool, carrying state across turns, and recovering from tool failures. We map the infrastructure shared bythese families, show how their scoring contracts differ, and explain why scaffold effects, oracle design,environment drift, schema sensitivity, and trace policy can change the scientific meaning of a result. Thepaper argues that harnesses are no longer peripheral implementation details. They define what the taskis, what the model is allowed to do, how evidence is scored, and whether a result can be audited or cited.We provide a layered taxonomy, a literature stem map, family-specific failure analyses, guidance for in-terpreting leaderboard scores, and a reporting agenda for model developers, benchmark maintainers, andresearchers building agentic systems. Fonte: https://api.crossref.org/works/10.20944%2Fpreprints202609.0236.v1 — metadati Crossref, consultati il 2026-10-06. Interrogazione: «large language model agents tool use».