Metraipolis › archive › dc_db3232aca3de

ElecSafety: Diagnosing Large Language Model Safety Judgments for Microcontroller Boards — arXiv:2610.06905

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.06905
License (as archived): CC0-1.0
Archived: 2026-10-07T11:25:34.548Z
SHA-256 of the archived text: 81d757352dbd0ec5fdefde47464bdd12d985999bb0bfd9d8898ede4007cb60eb
Stable URL of this page: https://metraipolis.com/documento/dc_db3232aca3de

How to cite

ElecSafety: Diagnosing Large Language Model Safety Judgments for Microcontroller Boards — arXiv:2610.06905 — arXiv (en), https://arxiv.org/abs/2610.06905 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_db3232aca3de (Metraipolis archive dc_db3232aca3de, sha256 81d757352dbd0ec5fdefde47464bdd12d985999bb0bfd9d8898ede4007cb60eb)

BibTeX

@misc{dcdb3232aca3de,
  title = {ElecSafety: Diagnosing Large Language Model Safety Judgments for Microcontroller Boards — arXiv:2610.06905},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.06905},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_db3232aca3de; sha256 81d757352dbd0ec5fdefde47464bdd12d985999bb0bfd9d8898ede4007cb60eb},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.06905. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) progressively support embedded hardware development, but a plausible recommendation may pose safety risks when acting on the physical hardware. Because electrical safety can be different between every microcontroller board, a practical question is raised: can LLMs have the capability to make the right decision on whether a proposed user operation is safe for the board? Although benchmarks for embedded development exist, they primarily target code generation and hardware design tasks. This highlights the lack of tested board-specific judgment regarding electrical safety. To address this gap, we introduce the ElecSafety benchmark to evaluate whether an LLM can respond with the correct label and derive that label from the proper manufacturer constraints and rules. We extract these constraints from vendor datasheets and hardware manuals. Each scenario is defined by a decisive condition and a gold label (safe, hazardous, or cannot determine) before translating the scenario into natural language. We categorize the scenarios into violation and compliant cases, board-swap, near-boundary, and missing-information. We then evaluated the scenario on six open-weight LLMs under various conditions of rule access and token limitations. The finding shows board rules can improve accuracy, whereas limiting the token has a smaller impact. Even the best configuration remains unreliable on near-boundary and missing-information scenarios, and a correct label often rests on the inaccurate constraints.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.AR, nuovi lavori del giorno).
Categorie arXiv: Hardware Architecture (cs.AR).
Pagina del lavoro: https://arxiv.org/abs/2610.06905 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:16:09.451Z dalla pagina https://arxiv.org/list/cs.AR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).