Metraipolis › archive › dc_0c660bbd88d2

Predictive Safety Guardrails Do Not Always Make LLM Agents Safer — arXiv:2611.02815

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.02815
License (as archived)
CC0-1.0
Archived
2026-11-03T10:44:05.534Z
SHA-256 of the archived text
f1464c8b6d0b776ef24ec188dc22df3ac86fcd4151477af290ecd8a872b792ef
Stable URL of this page
https://metraipolis.com/documento/dc_0c660bbd88d2

How to cite

Predictive Safety Guardrails Do Not Always Make LLM Agents Safer — arXiv:2611.02815 — arXiv (en), https://arxiv.org/abs/2611.02815 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0c660bbd88d2 (Metraipolis archive dc_0c660bbd88d2, sha256 f1464c8b6d0b776ef24ec188dc22df3ac86fcd4151477af290ecd8a872b792ef)
BibTeX
@misc{dc0c660bbd88d2,
  title = {Predictive Safety Guardrails Do Not Always Make LLM Agents Safer — arXiv:2611.02815},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02815},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0c660bbd88d2; sha256 f1464c8b6d0b776ef24ec188dc22df3ac86fcd4151477af290ecd8a872b792ef},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02815. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language model agents increasingly take consequential actions in digital and physical environments, where safety risks may emerge only after multiple interaction steps. This has motivated world-model-driven predictive safety guardrails, which anticipate future consequences before execution. However, existing evaluations largely focus on end-to-end safety outcomes, leaving unclear whether safety gains arise from reliable prediction and faithful use of predicted risks. In this work, we systematically diagnose the prediction-to-action pipeline along three dimensions: how world-model predictions are integrated into decision making, whether the predictions themselves are reliable, and whether their content is faithfully used. Across seven policy models and two safety environments, we find that stronger prediction involvement does not yield monotonic safety gains and can reduce task completion while substantially increasing inference cost. We further find that world-model predictions remain insufficiently reliable, especially over longer horizons, where safety prediction degrades substantially relative to near-term forecasting. Finally, we find that safety gains can persist even under corrupted predictions, while locally verified predictions still leave subsequent safety violations, showing that predictive content is not faithfully incorporated into the decision process. Together, these findings expose a fundamental prediction-to-action gap in current predictive guardrails: producing more or seemingly accurate foresight is insufficient unless that foresight is both reliable and faithfully translated into safer decisions.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.02815 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:39:50.313Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).