Metraipolis › archive › dc_4dcdb5b3aca2
ToolBottleneckBench: Measuring Tool-Agent Performance under Controlled Pressures and Fixed Harness Budgets — arXiv:2610.21098
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.21098
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T09:06:59.438Z
- SHA-256 of the archived text
a0e83b8809bf0ee5f054bdee25c2f8caac73859416c488b56fc867319994094c- Stable URL of this page
https://metraipolis.com/documento/dc_4dcdb5b3aca2
How to cite
ToolBottleneckBench: Measuring Tool-Agent Performance under Controlled Pressures and Fixed Harness Budgets — arXiv:2610.21098 — arXiv (en), https://arxiv.org/abs/2610.21098 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4dcdb5b3aca2 (Metraipolis archive dc_4dcdb5b3aca2, sha256 a0e83b8809bf0ee5f054bdee25c2f8caac73859416c488b56fc867319994094c)
BibTeX
@misc{dc4dcdb5b3aca2,
title = {ToolBottleneckBench: Measuring Tool-Agent Performance under Controlled Pressures and Fixed Harness Budgets — arXiv:2610.21098},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.21098},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4dcdb5b3aca2; sha256 a0e83b8809bf0ee5f054bdee25c2f8caac73859416c488b56fc867319994094c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21098. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
The success score of a tool agent reflects both its model and the budgets of the harness that runs it. We present ToolBottleneckBench, a benchmark that measures tool-agent performance under controlled pressures and fixed harness budgets. Synthetic enterprise tasks stay fixed while one pressure varies: the registered tool catalog, the visible tool menu, the certified workflow, or the encoding of returned records. An exact grader records the first unmet obligation of each failed episode and the harness limit, if any, that ended it. Across 21 model and harness configurations, the visible menu is the only one of the three pressures that act on the model with a large penalty, and this penalty tracks truncation at the completion cap. Raising the cap sixteenfold removes most of the penalty in the truncation-heavy configurations. Most failures end at a harness limit, and the failure record identifies the assigned pressure no more reliably than a count of tool calls. The repairs that help change which tools reach the model or who issues the calls, and only two repairs, neither improving model behavior, beat every configuration switch. We recommend reporting pressure effects with the harness budgets under which they were measured. We release the benchmark at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2610.21098 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T08:44:17.906Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).