Metraipolis › archive › dc_79435cd1912e

SCRIPTIOC-BENCH: A Benchmark for Recognizing Actionable Threat Intelligence from Script-Based Malware using LLMs — arXiv:2609.06149

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.06149
License (as archived): CC0-1.0
Archived: 2026-10-16T13:40:06.790Z
SHA-256 of the archived text: e323cf7295da3f83b43b5b75f4f0d72acccd96dd244da44a94fd1dac22119856
Stable URL of this page: https://metraipolis.com/documento/dc_79435cd1912e

How to cite

SCRIPTIOC-BENCH: A Benchmark for Recognizing Actionable Threat Intelligence from Script-Based Malware using LLMs — arXiv:2609.06149 — arXiv (en), https://arxiv.org/abs/2609.06149 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_79435cd1912e (Metraipolis archive dc_79435cd1912e, sha256 e323cf7295da3f83b43b5b75f4f0d72acccd96dd244da44a94fd1dac22119856)

BibTeX

@misc{dc79435cd1912e,
  title = {SCRIPTIOC-BENCH: A Benchmark for Recognizing Actionable Threat Intelligence from Script-Based Malware using LLMs — arXiv:2609.06149},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.06149},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_79435cd1912e; sha256 e323cf7295da3f83b43b5b75f4f0d72acccd96dd244da44a94fd1dac22119856},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.06149. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Script-based malware remains a prevalent attack technique. These scripts often contain indicators of compromise (IOCs) that provide actionable threat intelligence. However, statically recovering such indicators is challenging, as relevant values may be dispersed or transformed within code. Although large language models (LLMs) have shown promise in security analysis, their ability to recover IOCs from malicious scripts remains underexplored. We present SCRIPTIOC-BENCH, a benchmark for measuring static IOC extraction capability on real-world malicious scripts. The benchmark comprises 634 manually verified JavaScript, PowerShell, and VBScript malware samples covering four IOC types (URLs, domains, IP addresses, and filesystem artifacts). We further stratify ground-truth IOCs by recovery level, distinguishing directly exposed indicators from those requiring decoding or reconstruction. Using this benchmark, we evaluate a broad range of proprietary and open-weight LLMs and show that IOC recovery without execution remains challenging across model scales: the strongest model reaches only 65.4 F1. To characterize how recovery fails, we introduce a false-positive taxonomy and use it to compare the error profiles of the evaluated models. We further study two mitigations on a small open-weight model, deterministic string utilities and task-specific adaptation, finding that they provide complementary recovery gains, raise precision, and shift errors toward sample-grounded mismatches.

Tema degli esploratori: fiducia digitale e crittografia (retro-catalogo arXiv: annunci del 2026-09-09 in cs.CR).
Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.06149 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T13:09:48.407Z dalla pagina https://arxiv.org/catchup/cs.CR/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).