Metraipolis › archive › dc_d28decfaa314
EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing Evidence — arXiv:2610.31650
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.31650
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-23T15:14:41.979Z
- SHA-256 of the archived text
33f401426a6cadbe02b40a106a12ffa9d36c733f7f5892ed37f92ab6afa393cb- Stable URL of this page
https://metraipolis.com/documento/dc_d28decfaa314
How to cite
EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing Evidence — arXiv:2610.31650 — arXiv (en), https://arxiv.org/abs/2610.31650 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d28decfaa314 (Metraipolis archive dc_d28decfaa314, sha256 33f401426a6cadbe02b40a106a12ffa9d36c733f7f5892ed37f92ab6afa393cb)
BibTeX
@misc{dcd28decfaa314,
title = {EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing Evidence — arXiv:2610.31650},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.31650},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d28decfaa314; sha256 33f401426a6cadbe02b40a106a12ffa9d36c733f7f5892ed37f92ab6afa393cb},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.31650. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Tool-using language model agents rely on environmental facts to decide whether an action is permitted. When a tool response omits a safety-relevant fact, an agent may carry out a prohibited action even though the user request is benign and every returned statement is true. We introduce \bench{}, an executable benchmark for evaluating unsafe execution, safety boundaries, and evidence recovery in LLM agents under missing evidence. It contains 300 matched unsafe/safe task pairs across six domains. By systematically suppressing critical facts, the benchmark measures unsafe execution under critical evidence omission and determines the minimum observed failure cardinality, including failures with full evidence and cases with no observed failure after exhaustive testing. An independent audit tool tests whether agents seek missing facts and then withhold prohibited actions while completing permissible tasks. The benchmark reveals substantial variation across the ten models in omission robustness, evidence acquisition, and appropriate action. Resistance to missing evidence does not necessarily translate into useful recovery. Recovery access reduces unsafe execution for all ten models, but agents differ in whether they actively obtain the missing facts and use them to act appropriately in both worlds.
Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.31650 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-23T15:09:08.475Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).