Metraipolis › archive › dc_86cd74574c5d
CyberClear: A Benchmark for LLM Agent Systems on APT Attack Chain Provenance — arXiv:2609.32424
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.32424
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T21:00:12.483Z
- SHA-256 of the archived text
3e16917878912ca176fb77493d905e24b786e357bdb4c51a89d3aa8ff48ad4c6- Stable URL of this page
https://metraipolis.com/documento/dc_86cd74574c5d
How to cite
CyberClear: A Benchmark for LLM Agent Systems on APT Attack Chain Provenance — arXiv:2609.32424 — arXiv (en), https://arxiv.org/abs/2609.32424 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_86cd74574c5d (Metraipolis archive dc_86cd74574c5d, sha256 3e16917878912ca176fb77493d905e24b786e357bdb4c51a89d3aa8ff48ad4c6)
BibTeX
@misc{dc86cd74574c5d,
title = {CyberClear: A Benchmark for LLM Agent Systems on APT Attack Chain Provenance — arXiv:2609.32424},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.32424},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_86cd74574c5d; sha256 3e16917878912ca176fb77493d905e24b786e357bdb4c51a89d3aa8ff48ad4c6},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32424. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language model agents have demonstrated promising capabilities in cybersecurity tasks, yet their ability to reconstruct complete Advanced Persistent Threat attack campaigns from complex security logs remains largely unexplored. Existing cybersecurity benchmarks for agents mainly focus on vulnerability discovery, exploitation, and security analysis tasks, leaving the evaluation of attack chain provenance under realistic security logs insufficiently studied. To address this gap, we introduce CyberClear, a benchmark for evaluating LLM agents and advanced agent systems on APT attack chain provenance from long-context security logs. CyberClear covers both single-step attacks and multi-stage attack chains, requiring agents to identify attack evidence, infer attack progression, and generate provenance graphs containing entities, causal relationships, MITRE ATT&CK techniques, and forensic evidence. To enable comprehensive evaluation, we develop an evaluation method tailored to APT attack chain provenance. Unlike conventional text similarity metrics that focus on surface-level matching, our evaluation examines whether reconstructed graphs preserve the semantics of attack chains across single-step behavior correctness, multi-step behavior identification, temporal and causal consistency, entity and relationship fidelity, and overall attack narrative consistency. Advanced multi-agent systems powered by state-of-the-art LLMs still struggle on CyberClear, motivating us to propose CyberProvenance, an agent cyber harness designed for multi-agents that augments LLM agents with evidence accumulation, execution-based validation, and feedback-guided refinement mechanisms for reliable attack-chain provenance. Extensive evaluations on CyberClear demonstrate the effectiveness of CyberProvenance in improving evidence reasoning, execution-grounded validation, and complete APT attack chain reconstruction. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI). Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.32424 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:29:02.336Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).