Metraipolis › archive › dc_e8bb7cf7849d
APIFlow-Bench: Measuring Whether Agents Survive Long, Dependent API Workflows — arXiv:2608.29128
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.29128
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T07:44:05.923Z
- SHA-256 of the archived text
e85a1581daf2315b887c12827d2c3949f86696ea3a94452044812feb22908f7f- Stable URL of this page
https://metraipolis.com/documento/dc_e8bb7cf7849d
How to cite
APIFlow-Bench: Measuring Whether Agents Survive Long, Dependent API Workflows — arXiv:2608.29128 — arXiv (en), https://arxiv.org/abs/2608.29128 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e8bb7cf7849d (Metraipolis archive dc_e8bb7cf7849d, sha256 e85a1581daf2315b887c12827d2c3949f86696ea3a94452044812feb22908f7f)
BibTeX
@misc{dce8bb7cf7849d,
title = {APIFlow-Bench: Measuring Whether Agents Survive Long, Dependent API Workflows — arXiv:2608.29128},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.29128},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e8bb7cf7849d; sha256 e85a1581daf2315b887c12827d2c3949f86696ea3a94452044812feb22908f7f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.29128. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Tool-using agents are commonly evaluated by a single bit: whether an end-to-end workflow completed. This metric fails to distinguish failures that matter in production, such as expired credentials, malformed payloads, or correct execution followed by incorrect final delivery. We introduce APIFlow-Bench, a fully auditable benchmark for long-horizon, dependent REST-API workflows that decomposes performance into seven engineering capabilities and requires agents to produce answers supported by the actual call path. We generate synthetic API worlds forward, subtask by subtask; each subtask is admitted only after a zero-LLM self-test triad verifies its grader and an oracle establishes solvability, and an adversarial audit identified and fixed six grader exploits. Grading is deterministic and provenance-sensitive: a state check traces a mock-minted canary through the API data flow to the response the answer must originate from, and a typed answer card is verified field by field. We release all answer keys and 44,362 unredacted execution transcripts. Across 19 frontier and open-weight models under one neutral scaffold, we find: (1) longer dependency chains degrade success, from 93% on individual subtasks to 74% on clean 20-subtask chains and 61% when including the 8% of chain trials that a model-consensus screen flags as passed by no model; (2) reliability separates models more than best-case capability, with best-of-five spanning seven points but all-five-of-five reliability spanning 44 points; (3) the independent-error account of compounding failure does not fit the data: pass rates on 20-subtask chains are 33 percentage points above the product of subtask-level rates, and on the clean slice 77% of failing runs reached the correct final state and failed only at delivery. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG); Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2608.29128 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T07:28:26.517Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).