Metraipolis › archive › dc_1ae878ed91fa
SecLog-Bench: A Scalable Benchmark for Cybersecurity Log Reasoning with Scaffolded Evidence — arXiv:2610.31000
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.31000
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-25T08:34:47.345Z
- SHA-256 of the archived text
bf92ea4e089f770477126afaee08bc0bc62580163c6582f405149b116fbea18d- Stable URL of this page
https://metraipolis.com/documento/dc_1ae878ed91fa
How to cite
SecLog-Bench: A Scalable Benchmark for Cybersecurity Log Reasoning with Scaffolded Evidence — arXiv:2610.31000 — arXiv (en), https://arxiv.org/abs/2610.31000 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1ae878ed91fa (Metraipolis archive dc_1ae878ed91fa, sha256 bf92ea4e089f770477126afaee08bc0bc62580163c6582f405149b116fbea18d)
BibTeX
@misc{dc1ae878ed91fa,
title = {SecLog-Bench: A Scalable Benchmark for Cybersecurity Log Reasoning with Scaffolded Evidence — arXiv:2610.31000},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.31000},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1ae878ed91fa; sha256 bf92ea4e089f770477126afaee08bc0bc62580163c6582f405149b116fbea18d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.31000. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) have shown strong potential for cybersecurity, leading to growing interest in LLM-based automated cybersecurity log analysis. However, current datasets (1) are limited in scale; (2) rely mainly on single-source evidence; and (3) have only binary or multi-class attack labels, without dense annotations of the reasoning process. To address these limitations, SecLog-Bench has three key features. First, we build an extensible simulation platform that covers 29 attack scenarios, supports repeated execution and the integration of new scenarios, and enables scalable generation of cybersecurity logs with attack-stage labels. Second, we use multi-source logs, including audit logs, application logs, and network traffic, to jointly analyze cyber attacks. Third, we annotate each sample with the essential context for cybersecurity log reasoning and a structured step-by-step evidence chain. We evaluate eight mainstream LLMs on this benchmark for post-detection attack-progress reasoning. Results show that, when provided with scaffolded evidence, current LLMs exhibit modest capability in attack-progress reasoning, with the best model achieving only 44.44% accuracy and 45.07% Macro-F1. Their reasoning remains unreliable; i.e., the highest Stage Reasoning F1 (SRF1) score is only 54.87%. Moreover, even correct predictions may be accompanied by flawed reasoning, and reasoning quality is often misaligned with prediction accuracy. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.31000 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-25T08:31:13.214Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).