Metraipolis › archive › dc_89fe98015f38
CESBench: Benchmarking Large Language Models on Cryptographic Engineering Security for IoT Devices — arXiv:2609.21344
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.21344
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-10T01:07:56.857Z
- SHA-256 of the archived text
2c025b6550285dcb5537306833dcb92278b0636bc4c831f13b657c79b2462e02- Stable URL of this page
https://metraipolis.com/documento/dc_89fe98015f38
How to cite
CESBench: Benchmarking Large Language Models on Cryptographic Engineering Security for IoT Devices — arXiv:2609.21344 — arXiv (en), https://arxiv.org/abs/2609.21344 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_89fe98015f38 (Metraipolis archive dc_89fe98015f38, sha256 2c025b6550285dcb5537306833dcb92278b0636bc4c831f13b657c79b2462e02)
BibTeX
@misc{dc89fe98015f38,
title = {CESBench: Benchmarking Large Language Models on Cryptographic Engineering Security for IoT Devices — arXiv:2609.21344},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.21344},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_89fe98015f38; sha256 2c025b6550285dcb5537306833dcb92278b0636bc4c831f13b657c79b2462e02},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.21344. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
For Internet of Things (IoT) devices, a secure algorithm alone is not enough: an attacker with physical access can attack the implementation directly, and its flaws are hard to fix once deployed. Large language models (LLMs) are now used to build and analyze such implementations. LLM benchmarks exist for cryptography and general cybersecurity, but none covers cryptographic engineering. In this paper, we present CESBench, 380 expert-written items across six sub-domains of cryptographic engineering security for IoT devices: side-channel, fault injection, implementation, countermeasures, evaluation, and integration. Four task types target different competences: 209 multiple-choice items test recall, 67 judgment items require a security verdict and its justification, 63 scenario items require an engineering diagnosis, and 41 code tasks are graded by 572 test cases. To validate the benchmark, 11 open-weight and proprietary LLMs answer every item. Multiple-choice and code responses are scored automatically, and judgment and scenario responses by an LLM judge, whose scores are checked against a second judge from another model family and human re-scoring. Composite scores range from 54.4% to 83.6%. The top score on each task type is 98.6% for multiple choice, 95.1% for code, and 88.4% for scenario diagnosis, but only 58.8% for judgment. Across models, 88.5% of verdicts are correct, yet their justifications earn only 53.4% of the rubric marks. Multiple choice is near its ceiling for the strongest models and most code tasks are solved, whereas justifying a security verdict remains the weakest competence. The benchmark, prompts, and per-item results are public. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-21 in cs.AI). Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.21344 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-10T00:23:42.528Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-21?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).