Metraipolis › archive › dc_d06e392aecb6

EcoAgent-Bench: Evaluating Economic Decision-Making in Budget-Constrained LLM Agents — arXiv:2608.05519

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.05519
License (as archived)
CC0-1.0
Archived
2026-10-25T21:30:50.227Z
SHA-256 of the archived text
4f2440f1f1fd3e2fd823672b57bd5bf5a1f12d48dd14bd2929cf40399ed1702f
Stable URL of this page
https://metraipolis.com/documento/dc_d06e392aecb6

How to cite

EcoAgent-Bench: Evaluating Economic Decision-Making in Budget-Constrained LLM Agents — arXiv:2608.05519 — arXiv (en), https://arxiv.org/abs/2608.05519 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d06e392aecb6 (Metraipolis archive dc_d06e392aecb6, sha256 4f2440f1f1fd3e2fd823672b57bd5bf5a1f12d48dd14bd2929cf40399ed1702f)
BibTeX
@misc{dcd06e392aecb6,
  title = {EcoAgent-Bench: Evaluating Economic Decision-Making in Budget-Constrained LLM Agents — arXiv:2608.05519},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.05519},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d06e392aecb6; sha256 4f2440f1f1fd3e2fd823672b57bd5bf5a1f12d48dd14bd2929cf40399ed1702f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.05519. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Agent benchmarks usually measure task completion and treat resource use as an auxiliary statistic. In deployment, however, the choice among a local lookup, broad search, composite research tool, stronger model, or human escalation is part of the task itself. We introduce EcoAgent-Bench, in which every task specifies priced actions and an explicit budget. Its 304 real-derived tasks span five families adapted from GAIA, HotpotQA, and MuSiQue, and test four decisions: avoiding unnecessary escalation, escalating when local evidence is insufficient, selecting a model tier, and stopping on unsupported premises. We evaluate seven LLM agents in tool-API and workspace-CLI settings, together with four oracle scripted controls. Micro-averaged accuracy rewards one-sided policies: always-escalate controls achieve high micro success while failing save-oriented tasks. We therefore also report an economic-consistency score (the worse of accuracy on upgrade-oriented and save-oriented family groups) which exposes this failure. Tool-API agents attain only 3.9-24.0% micro strict success (at most 7.3% economic consistency), often either stopping before warranted escalation or overspending on cheap tasks. A threshold-crossing budget sweep changes GPT-5.4's escalation rate from 0% to only 3%. These results show that completion under a budget and economical action selection are distinct properties. We release the task bundle, transformation pipeline, frozen evaluation environments, and integrity-bound result artifacts needed to study both.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-07 in cs.CL).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.05519 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-25T21:24:38.252Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).