Metraipolis › archive › dc_04ccdedde245

From Investigation Failures to Reliable SOC Agents: Understanding and Improving LLM-Based Alert Triage — arXiv:2610.10608

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.10608
License (as archived): CC0-1.0
Archived: 2026-10-09T05:46:54.516Z
SHA-256 of the archived text: c39087c9a396f230085cfaf62bd5caf0506d38195b2d6b33dbd0ac9acfa8950e
Stable URL of this page: https://metraipolis.com/documento/dc_04ccdedde245

How to cite

From Investigation Failures to Reliable SOC Agents: Understanding and Improving LLM-Based Alert Triage — arXiv:2610.10608 — arXiv (en), https://arxiv.org/abs/2610.10608 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_04ccdedde245 (Metraipolis archive dc_04ccdedde245, sha256 c39087c9a396f230085cfaf62bd5caf0506d38195b2d6b33dbd0ac9acfa8950e)

BibTeX

@misc{dc04ccdedde245,
  title = {From Investigation Failures to Reliable SOC Agents: Understanding and Improving LLM-Based Alert Triage — arXiv:2610.10608},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.10608},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_04ccdedde245; sha256 c39087c9a396f230085cfaf62bd5caf0506d38195b2d6b33dbd0ac9acfa8950e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.10608. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Security operations centers (SOCs) must triage large volumes of alerts, most of which are benign, while missed attacks can remain uninvestigated. Tool-using large language model (LLM) agents can retrieve evidence during triage, but it remains unclear how reasoning strategies determine what to gather and when an investigation is sufficient to close an alert. We study five representative approaches spanning single-pass tool use, iterative retrieval, sampled investigations, self-review, and explicit verification. To support this study, we build ALERT-BENCH, an interactive benchmark that replays enterprise telemetry through a live SIEM and requires each system to retrieve evidence. Across 1,247 alerts from a multi-stage attack scenario, every approach missed at least 40.4% of attack-related alerts. Trace analysis shows that attack alerts are more likely to be dismissed when searches return no records, same-context review has negative net correction, and dismissal receives no consistently stronger investigation than escalation. Based on these findings, we further design AIDA (Adversarial Investigation and Dialectical Analysis), a multi-agent framework that requires an explicit proposed decision before independent challenge and stronger evidentiary requirements before dismissal. AIDA preserves investigation history in an append-only Investigation Ledger and keeps the challenge in a separate reasoning context. A separate Judge adjudicates the proposed decision and challenge against evidence, resolving the alert or requesting another round when evidence is missing. On the same alerts, AIDA achieves an F1 score of 0.958, compared with 0.371-0.744 for the studied approaches, and reduces the false-negative rate from 40.4% to 3.1% while escalating 18.4% of alerts to analysts. These results show that structuring evidence retrieval and decision review can substantially improve agentic SOC triage.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.MA, nuovi lavori del giorno).
Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA).
Pagina del lavoro: https://arxiv.org/abs/2610.10608 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T05:46:44.100Z dalla pagina https://arxiv.org/list/cs.MA/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).