Metraipolis › archive › dc_2d60ae90707d

Predictive Safety Guardrails Do Not Always Make LLM Agents Safer — arXiv:2611.02815

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.02815
License (as archived)
CC0-1.0
Archived
2026-11-03T10:32:58.638Z
SHA-256 of the archived text
b74996569e6dfb26e248e1dafd672b66199b15c1a9de49db491d1594b86ad5d8
Stable URL of this page
https://metraipolis.com/documento/dc_2d60ae90707d

How to cite

Predictive Safety Guardrails Do Not Always Make LLM Agents Safer — arXiv:2611.02815 — arXiv (en), https://arxiv.org/abs/2611.02815 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2d60ae90707d (Metraipolis archive dc_2d60ae90707d, sha256 b74996569e6dfb26e248e1dafd672b66199b15c1a9de49db491d1594b86ad5d8)
BibTeX
@misc{dc2d60ae90707d,
  title = {Predictive Safety Guardrails Do Not Always Make LLM Agents Safer — arXiv:2611.02815},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02815},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2d60ae90707d; sha256 b74996569e6dfb26e248e1dafd672b66199b15c1a9de49db491d1594b86ad5d8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02815. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language model agents increasingly take consequential actions in digital and physical environments, where safety risks may emerge only after multiple interaction steps. This has motivated world-model-driven predictive safety guardrails, which anticipate future consequences before execution. However, existing evaluations largely focus on end-to-end safety outcomes, leaving unclear whether safety gains arise from reliable prediction and faithful use of predicted risks. In this work, we systematically diagnose the prediction-to-action pipeline along three dimensions: how world-model predictions are integrated into decision making, whether the predictions themselves are reliable, and whether their content is faithfully used. Across seven policy models and two safety environments, we find that stronger prediction involvement does not yield monotonic safety gains and can reduce task completion while substantially increasing inference cost. We further find that world-model predictions remain insufficiently reliable, especially over longer horizons, where safety prediction degrades substantially relative to near-term forecasting. Finally, we find that safety gains can persist even under corrupted predictions, while locally verified predictions still leave subsequent safety violations, showing that predictive content is not faithfully incorporated into the decision process. Together, these findings expose a fundamental prediction-to-action gap in current predictive guardrails: producing more or seemingly accurate foresight is insufficient unless that foresight is both reliable and faithfully translated into safer decisions.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.02815 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:29:48.595Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).