Metraipolis › archive › dc_dc8c28dea0da
SecLog-Bench: A Scalable Benchmark for Cybersecurity Log Reasoning with Scaffolded Evidence — arXiv:2610.31000
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.31000
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-23T15:15:15.453Z
- SHA-256 of the archived text
285a51ed488d076df4bc8cff4828241bb2b50c993d3585ea1b924d5ab9a9d8ef- Stable URL of this page
https://metraipolis.com/documento/dc_dc8c28dea0da
How to cite
SecLog-Bench: A Scalable Benchmark for Cybersecurity Log Reasoning with Scaffolded Evidence — arXiv:2610.31000 — arXiv (en), https://arxiv.org/abs/2610.31000 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dc8c28dea0da (Metraipolis archive dc_dc8c28dea0da, sha256 285a51ed488d076df4bc8cff4828241bb2b50c993d3585ea1b924d5ab9a9d8ef)
BibTeX
@misc{dcdc8c28dea0da,
title = {SecLog-Bench: A Scalable Benchmark for Cybersecurity Log Reasoning with Scaffolded Evidence — arXiv:2610.31000},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.31000},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dc8c28dea0da; sha256 285a51ed488d076df4bc8cff4828241bb2b50c993d3585ea1b924d5ab9a9d8ef},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.31000. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) have shown strong potential for cybersecurity, leading to growing interest in LLM-based automated cybersecurity log analysis. However, current datasets (1) are limited in scale; (2) rely mainly on single-source evidence; and (3) have only binary or multi-class attack labels, without dense annotations of the reasoning process. To address these limitations, SecLog-Bench has three key features. First, we build an extensible simulation platform that covers 29 attack scenarios, supports repeated execution and the integration of new scenarios, and enables scalable generation of cybersecurity logs with attack-stage labels. Second, we use multi-source logs, including audit logs, application logs, and network traffic, to jointly analyze cyber attacks. Third, we annotate each sample with the essential context for cybersecurity log reasoning and a structured step-by-step evidence chain. We evaluate eight mainstream LLMs on this benchmark for post-detection attack-progress reasoning. Results show that, when provided with scaffolded evidence, current LLMs exhibit modest capability in attack-progress reasoning, with the best model achieving only 44.44% accuracy and 45.07% Macro-F1. Their reasoning remains unreliable; i.e., the highest Stage Reasoning F1 (SRF1) score is only 54.87%. Moreover, even correct predictions may be accompanied by flawed reasoning, and reasoning quality is often misaligned with prediction accuracy. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.31000 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-23T15:09:08.577Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).