Metraipolis › archive › dc_98ee0bb75546
SAFARI: An Industrial Benchmark for LLM-Assisted Hazard Analysis and Risk Assessment — arXiv:2609.20584
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.20584
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-10T03:48:24.107Z
- SHA-256 of the archived text
e145f7efffc9f6beefef6257c724578dd3a4b266c202c35e90cd1722e044add6- Stable URL of this page
https://metraipolis.com/documento/dc_98ee0bb75546
How to cite
SAFARI: An Industrial Benchmark for LLM-Assisted Hazard Analysis and Risk Assessment — arXiv:2609.20584 — arXiv (en), https://arxiv.org/abs/2609.20584 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_98ee0bb75546 (Metraipolis archive dc_98ee0bb75546, sha256 e145f7efffc9f6beefef6257c724578dd3a4b266c202c35e90cd1722e044add6)
BibTeX
@misc{dc98ee0bb75546,
title = {SAFARI: An Industrial Benchmark for LLM-Assisted Hazard Analysis and Risk Assessment — arXiv:2609.20584},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.20584},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_98ee0bb75546; sha256 e145f7efffc9f6beefef6257c724578dd3a4b266c202c35e90cd1722e044add6},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.20584. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) are increasingly considered for safety-critical engineering, yet their reliability in regulated functional-safety workflows remains underexplored. We introduce SAFARI (Safety-Aware Functional Automotive Risk Inference), the first industrial benchmark for LLM-assisted automotive Hazard Analysis and Risk Assessment (HARA) under ISO 26262. It contains 3,000 de-identified industrial HARA cases and evaluates two coupled tasks: open-ended hazard analysis and standards-grounded risk assessment. To evaluate open-ended HARA artifacts, we propose the first reference-anchored LLM-as-a-judge protocol with high expert correlation. Experiments with nine frontier LLMs show that models often produce plausible hazard narratives but remain weak at ISO 26262 risk classification, with the best ASIL macro-F1 reaching only 0.261. Chain-of-Thought prompting provides limited benefit and often degrades categorical risk assessment. Error analysis further localizes major failures to scenario-critical context omissions during hazard generation and to controllability misjudgments during risk assessment, indicating where expert oversight should be concentrated. The dataset can be obtained from this https URL . Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-18 in cs.CL). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.20584 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-10T03:24:08.166Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).