Metraipolis › archive › dc_f26fb7b2f1bf
Beyond Infraction Scores: Failure-Boundary Profiles for Closed-Loop Evaluation of Driving and Robot Navigation Agents — arXiv:2611.06453
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.06453
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-05T18:26:33.654Z
- SHA-256 of the archived text
df3f219c211c8359da6deccde47efe82b9d2f7d3e4a409cba5b24217948f498c- Stable URL of this page
https://metraipolis.com/documento/dc_f26fb7b2f1bf
How to cite
Beyond Infraction Scores: Failure-Boundary Profiles for Closed-Loop Evaluation of Driving and Robot Navigation Agents — arXiv:2611.06453 — arXiv (en), https://arxiv.org/abs/2611.06453 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f26fb7b2f1bf (Metraipolis archive dc_f26fb7b2f1bf, sha256 df3f219c211c8359da6deccde47efe82b9d2f7d3e4a409cba5b24217948f498c)
BibTeX
@misc{dcf26fb7b2f1bf,
title = {Beyond Infraction Scores: Failure-Boundary Profiles for Closed-Loop Evaluation of Driving and Robot Navigation Agents — arXiv:2611.06453},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.06453},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f26fb7b2f1bf; sha256 df3f219c211c8359da6deccde47efe82b9d2f7d3e4a409cba5b24217948f498c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.06453. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Closed-loop driving benchmarks in CARLA rank agents by a driving score that multiplies route completion by an infraction penalty. This score does not distinguish an agent that stops in front of an obstacle from one that pushes past it. In a corridor closed by two parked cars, for example, one agent that drives through with contact averages 52.2 on the blocked widths, while the agents that stop average between 37.8 and 40.6. We propose failure-boundary profiles, an evaluation in which each scenario family varies one measured factor, such as corridor width or the time gap in oncoming traffic, and each variant is certified from measurement as one where the agent should proceed or one where it should stop. Along this axis we estimate two curves per agent, commitment (the probability of attempting the passage) and competence (the probability of passing without contact), and locate where each crosses one half, with intervals that account for the simulator's run-to-run variance. The certified labels also give a summary metric, decision accuracy, which credits clean passes on passable variants and stops on blocked ones. We evaluate seven driving agents on seven CARLA families. The rule-based expert reaches a decision accuracy of 0.74 and TFv6, the highest-scoring learned agent, 0.60. Agents with similar driving scores fail in different ways, and agents that reliably stop at a blocked corridor still rarely wait for a gap in oncoming traffic. We also apply the method, unchanged, to a warehouse robot in Isaac Sim. The navigation policy we test stops at blocked gaps in only one of three families, and two fine-tuned versions of it pass more gaps cleanly but no longer stop at all. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.SE, nuovi lavori del giorno). Categorie arXiv: Robotics (cs.RO) ; Computer Vision and Pattern Recognition (cs.CV); Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2611.06453 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-05T18:16:03.888Z dalla pagina https://arxiv.org/list/cs.SE/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).