Metraipolis › archive › dc_fd6feaf89320

False Reassurance in LLM Severe-Weather Guidance: A Replayable Evaluation Against Real NWS Alerts — arXiv:2611.05290

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.05290
License (as archived): CC0-1.0
Archived: 2026-11-04T21:40:26.101Z
SHA-256 of the archived text: 4756ba8e67eca7ac78101771e36af487f37269347670f0cbc5cd72a5d3046799
Stable URL of this page: https://metraipolis.com/documento/dc_fd6feaf89320

How to cite

False Reassurance in LLM Severe-Weather Guidance: A Replayable Evaluation Against Real NWS Alerts — arXiv:2611.05290 — arXiv (en), https://arxiv.org/abs/2611.05290 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fd6feaf89320 (Metraipolis archive dc_fd6feaf89320, sha256 4756ba8e67eca7ac78101771e36af487f37269347670f0cbc5cd72a5d3046799)

BibTeX

@misc{dcfd6feaf89320,
  title = {False Reassurance in LLM Severe-Weather Guidance: A Replayable Evaluation Against Real NWS Alerts — arXiv:2611.05290},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.05290},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fd6feaf89320; sha256 4756ba8e67eca7ac78101771e36af487f37269347670f0cbc5cd72a5d3046799},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05290. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

An LLM assistant that cannot see official severe-weather alerts may tell users that conditions are calm while a warning is active. We call this false reassurance and measure it by replaying 117 frozen cases built from real U.S. National Weather Service (NWS) alerts (69 alert cases, 32 no-alert controls, 16 non-U.S. locations) through five conditions of a Claude-based preparedness pipeline that differ in alert access and prompt design. One rater hand-labeled all 342 alert-case responses under a seven-category scheme written after an initial binary pass, which we disclose. False reassurance was 0.0% for the production pipeline (0/66, 95% CI 0.0-5.5) and for the same alerts under a neutral prompt (0/69), 37.7% for a blind neutral prompt (26/69), and 44.9% when the blind prompt also told the model to disclose its blind spot (31/69). Every case was implicit; no response explicitly denied an alert. Real alerts removed false reassurance (paired McNemar p < 0.0001); the disclosure instruction did not (p = 0.40). An automated regex measure suggested the opposite and missed 53 of 57 true cases, so we do not use it. Alert data alone was not enough for safe behavior: with a neutral prompt the model raised some alarm on 87.5% of no-alert controls, against 0% under the production prompt, and a preregistered single-sentence ablation showed that one production instruction explains part of that gap. We also report two failures common to similar systems, NWS test products ingested as real alerts and a token budget that fails on complex cases, and we release the dataset, labels and code.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Human-Computer Interaction (cs.HC) ; Artificial Intelligence (cs.AI); Computers and Society (cs.CY).
Pagina del lavoro: https://arxiv.org/abs/2611.05290 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T21:35:49.718Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).