Metraipolis › archive › dc_34192adeea33

Forgeable Confirmation in Automated Computer Security Testing: Deterministic Rules versus AI Judges — arXiv:2609.24200

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.24200
License (as archived): CC0-1.0
Archived: 2026-10-09T07:01:11.886Z
SHA-256 of the archived text: c4b066acae7e787b9987ebb3a223c8abedfbbc9c225005fa081d50545fadfc6e
Stable URL of this page: https://metraipolis.com/documento/dc_34192adeea33

How to cite

Forgeable Confirmation in Automated Computer Security Testing: Deterministic Rules versus AI Judges — arXiv:2609.24200 — arXiv (en), https://arxiv.org/abs/2609.24200 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_34192adeea33 (Metraipolis archive dc_34192adeea33, sha256 c4b066acae7e787b9987ebb3a223c8abedfbbc9c225005fa081d50545fadfc6e)

BibTeX

@misc{dc34192adeea33,
  title = {Forgeable Confirmation in Automated Computer Security Testing: Deterministic Rules versus AI Judges — arXiv:2609.24200},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.24200},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_34192adeea33; sha256 c4b066acae7e787b9987ebb3a223c8abedfbbc9c225005fa081d50545fadfc6e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.24200. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

AI is increasingly used to automate computer security testing, and the tools must decide for themselves whether an attack succeeded. A finding that a deterministic rule confirms by observation is reported as fact, whereas one that an LLM judges exploitable is treated as an opinion. We ask whether the system under test can forge that confirmation. In offline security testing of a four-stage AI-assisted pipeline, nine of its fifteen confirmation mechanisms are forgeable, and forgeability is predicted entirely by whether the decision reads attacker-controlled data. We formalise this as an auditable attack surface and test it prospectively: on sixteen held-out mechanisms, predictions fixed before any attack separated forgeable from unforgeable mechanisms exactly, and across 12,203 mechanisms in public scanner templates the prediction was 99.9% accurate. Deterministic rules proved cheaper to forge than eight open-weight LLM judges, failing at 2% of attacker-controlled response content against a median of 50%. No implementation of one check was both robust and precise, and routing between a rule and an AI judge raised forgery to 99%. Moving the decisive evidence to a channel the attacker cannot write cuts attack success from 97% to 0%, and an escalate verdict recovers the sensitivity this costs. The protection fails when the scanned host is itself the adversary. The results bear on AI security agents and on benchmarks that score success by string matching.

Tema degli esploratori: fiducia digitale e crittografia (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CR).
Categorie arXiv: Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2609.24200 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:56:00.999Z dalla pagina https://arxiv.org/catchup/cs.CR/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).