Metraipolis › archive › dc_83b7ffd80dca

Safety Monitors Mostly Catch What the Model Already Refuses — arXiv:2609.05797

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.05797
License (as archived)
CC0-1.0
Archived
2026-10-16T12:34:45.178Z
SHA-256 of the archived text
178ee0709380d4fba7dfcf6d41bb3b2f191c70a3d29aee65a0dad2ef6739d29d
Stable URL of this page
https://metraipolis.com/documento/dc_83b7ffd80dca

How to cite

Safety Monitors Mostly Catch What the Model Already Refuses — arXiv:2609.05797 — arXiv (en), https://arxiv.org/abs/2609.05797 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_83b7ffd80dca (Metraipolis archive dc_83b7ffd80dca, sha256 178ee0709380d4fba7dfcf6d41bb3b2f191c70a3d29aee65a0dad2ef6739d29d)
BibTeX
@misc{dc83b7ffd80dca,
  title = {Safety Monitors Mostly Catch What the Model Already Refuses — arXiv:2609.05797},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.05797},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_83b7ffd80dca; sha256 178ee0709380d4fba7dfcf6d41bb3b2f191c70a3d29aee65a0dad2ef6739d29d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.05797. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Safety monitors are evaluated by recall on harmful prompts, regardless of whether the target model would answer them. Yet a monitor matters most on the prompts the model does answer. We measure recall on exactly those prompts, defined by sampling the target model and judging its responses. Across four text guards, two activation probes, and Latent Guard, recall at a 1% false positive rate falls sharply on this subset: at a common threshold, every monitor catches the requests the model refuses 1.1 to 6.4 times as often as the requests it answers. Standard metrics hide this; AUROC stays above 0.85 for most monitors. Rewriting each request to be less explicit, with intent held fixed and verified, raises compliance 28-fold and lowers every monitor's flag rate. Of the requests newly answered after rewriting, 44 to 93% slip past the monitor, depending on which is used, and most of their completions are graded harmful. We trace the gap to explicitness itself. As wording softens with intent fixed, the target model's harm and refusal readings fall and it answers; the guards' harm readings fall too, and steering a guard along explicitness alone flips its verdict. Model and monitors miss the same prompts, and stacking monitors does not recover them. Fine-tuning a guard on the rewrites at every level of explicitness, on both sides of the label, raises recall on answered requests from .24 to .89 while transferring to unseen benchmarks; hard negatives, the natural alternative, teach the guard to discount indirect phrasing instead.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.05797 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T12:30:24.982Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).