Metraipolis › archive › dc_e218a0e8a346

ATOBench: Tracing How Autonomous Penetration-Testing Agents Verify Vulnerabilities When Target Evidence Lies — arXiv:2608.12996

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.12996
License (as archived)
CC0-1.0
Archived
2026-10-19T04:26:15.531Z
SHA-256 of the archived text
5cd0f836f8eca48856ac9b5bbe7476abd6f0d3e5b817811a17cccbd9de522c86
Stable URL of this page
https://metraipolis.com/documento/dc_e218a0e8a346

How to cite

ATOBench: Tracing How Autonomous Penetration-Testing Agents Verify Vulnerabilities When Target Evidence Lies — arXiv:2608.12996 — arXiv (en), https://arxiv.org/abs/2608.12996 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e218a0e8a346 (Metraipolis archive dc_e218a0e8a346, sha256 5cd0f836f8eca48856ac9b5bbe7476abd6f0d3e5b817811a17cccbd9de522c86)
BibTeX
@misc{dce218a0e8a346,
  title = {ATOBench: Tracing How Autonomous Penetration-Testing Agents Verify Vulnerabilities When Target Evidence Lies — arXiv:2608.12996},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.12996},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e218a0e8a346; sha256 5cd0f836f8eca48856ac9b5bbe7476abd6f0d3e5b817811a17cccbd9de522c86},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.12996. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Autonomous penetration-testing agents rely on target responses. These responses guide both subsequent actions and the final report. A deceptive response can therefore redirect both the attack trajectory and the agent's verification process. However, final reports reveal little about how an agent interprets conflicting evidence, changes course, decides to stop, or turns observations into a vulnerability claim. We introduce ATOBench, an evaluation framework that makes this verification process observable. ATOBench injects registered response transformations at runtime and pairs each transformed episode with a native episode under the same environment. Each pair is aligned at the first affected response. A source-linked reconstruction then follows later actions, evidence recovery, stopping, and report support. Three frozen observation contracts cover exploit proof, resource ownership, and reusable artifacts. We evaluate five model routes over 450 episodes. The analysis shows that increased activity can mask a broken verification chain, while successful recovery depends on finding usable evidence and preserving it through reporting. ATOBench turns deceptive target observations into a reproducible probe of evidence handling in autonomous penetration testing. This process-level view extends offensive pentest agent evaluation beyond final outcomes by revealing how untrusted observations shape actions, verification, and reporting.

Tema degli esploratori: fiducia digitale e crittografia (retro-catalogo arXiv: annunci del 2026-08-14 in cs.CR).
Categorie arXiv: Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2608.12996 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T04:20:00.560Z dalla pagina https://arxiv.org/catchup/cs.CR/2026-08-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).