Metraipolis › archive › dc_4c4533d94fee
BABA-Bench: Evaluating Long-Horizon Patient-Advocacy Agents — arXiv:2610.41076
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.41076
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T08:00:40.832Z
- SHA-256 of the archived text
8f4a417dd277ed7d575d6cd815329b77af4ceb881ea5e220dab26bfe3897334e- Stable URL of this page
https://metraipolis.com/documento/dc_4c4533d94fee
How to cite
BABA-Bench: Evaluating Long-Horizon Patient-Advocacy Agents — arXiv:2610.41076 — arXiv (en), https://arxiv.org/abs/2610.41076 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4c4533d94fee (Metraipolis archive dc_4c4533d94fee, sha256 8f4a417dd277ed7d575d6cd815329b77af4ceb881ea5e220dab26bfe3897334e)
BibTeX
@misc{dc4c4533d94fee,
title = {BABA-Bench: Evaluating Long-Horizon Patient-Advocacy Agents — arXiv:2610.41076},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.41076},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4c4533d94fee; sha256 8f4a417dd277ed7d575d6cd815329b77af4ceb881ea5e220dab26bfe3897334e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41076. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
An LLM agent that helps a patient get care must do two things. It must reach the correct outcome, and it must follow the necessary procedure. For example, the agent must get the patient's permission before it shares the patient's records. We introduce BABA-Bench, a benchmark of 47 interactive worlds. We built each world from a de-identified, real patient advocacy case that expert advocates reviewed. In each world, the agent must find hidden information and coordinate insurers, clinics, suppliers, pharmacies, and community organizations. The engine checks each action at the time the agent does it. It checks if the agent has permission, if the agent has evidence for the action, and if the action is prohibited. We evaluated 14 frontier models with three runs on each world, for a total of 1,974 runs. Agents often reached the correct outcome by an invalid route. Of the 921 runs that reached all required outcomes, 254 (28%) did not follow the necessary procedure. Claude Opus 5 reached all required outcomes in 81.6% of its runs, but only 52.5% of its runs passed all checks. When we gave two models a verified route, their pass rates increased from 29.8% to 91.5% and from 44.0% to 97.2%. For these models, the main problem is to find a route, not to execute it. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.41076 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T07:59:38.683Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).