Metraipolis › archive › dc_40de640b475f

Verdict Without the Rule: Diagnosing and Auditing Regulatory Rule Sensitivity in LLM Compliance Systems — arXiv:2610.12313

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.12313
License (as archived)
CC0-1.0
Archived
2026-10-09T05:50:38.703Z
SHA-256 of the archived text
a5611af5369a2eac81f79a71dae3676ce31bf47ae3e4a0d7c782ae296fb5f21b
Stable URL of this page
https://metraipolis.com/documento/dc_40de640b475f

How to cite

Verdict Without the Rule: Diagnosing and Auditing Regulatory Rule Sensitivity in LLM Compliance Systems — arXiv:2610.12313 — arXiv (en), https://arxiv.org/abs/2610.12313 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_40de640b475f (Metraipolis archive dc_40de640b475f, sha256 a5611af5369a2eac81f79a71dae3676ce31bf47ae3e4a0d7c782ae296fb5f21b)
BibTeX
@misc{dc40de640b475f,
  title = {Verdict Without the Rule: Diagnosing and Auditing Regulatory Rule Sensitivity in LLM Compliance Systems — arXiv:2610.12313},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.12313},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_40de640b475f; sha256 a5611af5369a2eac81f79a71dae3676ce31bf47ae3e4a0d7c782ae296fb5f21b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.12313. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language model compliance systems are deployed on the assumption that a verdict depends on the regulatory rule it is given. We test this directly across five models and 20 regulatory and platform-policy domains: delete, swap, or negate the governing rule while holding the case fixed, and check whether the verdict changes (OCS) or the model's internal representation of compliance shifts at all (ICS-delta). Neither moves much: models' verdicts are often invariant to substantial perturbations of the supplied rule, and the guard model, evaluated here under a custom-rule adaptation of its native taxonomy, is the least rule-sensitive and least accurate of the five, barely above chance (51%, versus 90-92% for general-purpose models). This reflects easy cases more than blanket neglect: on cases where deleting the rule changes a previously correct model prediction, models do track it closely. Neither better prompting nor direct intervention on the model's internal representations closes this gap. Accuracy alone does not establish that a compliance verdict is grounded in the supplied rule.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Computers and Society (cs.CY).
Pagina del lavoro: https://arxiv.org/abs/2610.12313 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T05:47:01.102Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).