Metraipolis › archive › dc_0fa96ebaa0ef

Evaluating System One Models for Agent Security Decisions: Reliability, Calibration, and Selective Automation — arXiv:2609.33401

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.33401
License (as archived): CC0-1.0
Archived: 2026-10-07T21:01:23.616Z
SHA-256 of the archived text: 0ba446bedaa7d5ef2e48edb6b23fdb5b544e14fac7510a62aa5e9e53bce21d8d
Stable URL of this page: https://metraipolis.com/documento/dc_0fa96ebaa0ef

How to cite

Evaluating System One Models for Agent Security Decisions: Reliability, Calibration, and Selective Automation — arXiv:2609.33401 — arXiv (en), https://arxiv.org/abs/2609.33401 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0fa96ebaa0ef (Metraipolis archive dc_0fa96ebaa0ef, sha256 0ba446bedaa7d5ef2e48edb6b23fdb5b544e14fac7510a62aa5e9e53bce21d8d)

BibTeX

@misc{dc0fa96ebaa0ef,
  title = {Evaluating System One Models for Agent Security Decisions: Reliability, Calibration, and Selective Automation — arXiv:2609.33401},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.33401},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0fa96ebaa0ef; sha256 0ba446bedaa7d5ef2e48edb6b23fdb5b544e14fac7510a62aa5e9e53bce21d8d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33401. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Agentic software connects language models to tools that modify files and interact with external services. Developers use model-based security checks to screen external content and user requests before agents act. System One models answer developer-defined questions with probabilities over predefined answers such as safe or unsafe, but classification accuracy alone does not establish whether these probabilities support reliable automation. We evaluate Jev, Laya, Decider, and Bespoke Nimble against specialized classifiers and language-model judges across prompt-injection detection, interaction-risk judgment, and harmful-request screening, examining decision accuracy, calibration, and selective automation. (1) High overall accuracy and low average calibration error can conceal attacks classified as safe with high confidence within particular groups. Developers should test candidate models on the intended security task and examine errors within relevant attack groups. (2) Policies selected under strict miss limits allow few test inputs automatically, and choosing allow and block thresholds separately increases automation mainly through additional blocks. Developers should verify limits on missed unsafe inputs and blocked benign inputs on independent data, and report the allowed and blocked fractions separately. (3) Comparing predictions on the same inputs shows that a language-model judge can detect unsafe inputs missed by a System One model, but can also repeat the System One model's confident mistakes and falsely flag benign inputs. Developers should test which missed unsafe inputs their review rule forwards, then measure the review model's misses and benign false alarms on those inputs.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI); Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2609.33401 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:02.639Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).