Metraipolis › archive › dc_bafef15c491f

CAPSafe: A Benchmark for Contextual Action and Plan Safety Reasoning in Embodied Agents — arXiv:2610.28381

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.28381
License (as archived)
CC0-1.0
Archived
2026-10-21T07:11:21.637Z
SHA-256 of the archived text
a3cf380ccf4f14f91f516f3d837f043c412aaa9b07ea2b811b654c4b5ce3e7ab
Stable URL of this page
https://metraipolis.com/documento/dc_bafef15c491f

How to cite

CAPSafe: A Benchmark for Contextual Action and Plan Safety Reasoning in Embodied Agents — arXiv:2610.28381 — arXiv (en), https://arxiv.org/abs/2610.28381 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bafef15c491f (Metraipolis archive dc_bafef15c491f, sha256 a3cf380ccf4f14f91f516f3d837f043c412aaa9b07ea2b811b654c4b5ce3e7ab)
BibTeX
@misc{dcbafef15c491f,
  title = {CAPSafe: A Benchmark for Contextual Action and Plan Safety Reasoning in Embodied Agents — arXiv:2610.28381},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.28381},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bafef15c491f; sha256 a3cf380ccf4f14f91f516f3d837f043c412aaa9b07ea2b811b654c4b5ce3e7ab},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.28381. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reasoning about safety is essential for embodied agents powered by large language models (LLMs). We introduce the Contextual Action and Plan Safety benchmark (CAPSafe), designed to evaluate safety reasoning at two levels: (1) an action-level dataset that tests whether a single action is safe given its environmental context, and (2) a plan-level dataset that evaluates whether agents can generate safe long-horizon plans for complex hazardous household tasks. CAPSafe models safety conditions where hazards are situational (inferred from object properties), multiple hazards can co-occur within the same environment, and each is annotated with a relative severity level for fine-grained analysis. Evaluating nine closed and open-source LLMs reveals a substantial gap between hazard recognition and safe long-horizon planning. Several models achieve high accuracy in recognizing unsafe actions (exceeding 90% in the best cases), yet safe long-horizon planning remains substantially harder. Gemini-3-pro achieves the highest safe success rate (78.19%), while other models remain below 65%. These results highlight that robust safety reasoning over long action sequences remains an open challenge.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Robotics (cs.RO) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.28381 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-21T07:05:58.181Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).