Metraipolis › archive › dc_40de640b475f
Verdict Without the Rule: Diagnosing and Auditing Regulatory Rule Sensitivity in LLM Compliance Systems — arXiv:2610.12313
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.12313
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T05:50:38.703Z
- SHA-256 of the archived text
a5611af5369a2eac81f79a71dae3676ce31bf47ae3e4a0d7c782ae296fb5f21b- Stable URL of this page
https://metraipolis.com/documento/dc_40de640b475f
How to cite
Verdict Without the Rule: Diagnosing and Auditing Regulatory Rule Sensitivity in LLM Compliance Systems — arXiv:2610.12313 — arXiv (en), https://arxiv.org/abs/2610.12313 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_40de640b475f (Metraipolis archive dc_40de640b475f, sha256 a5611af5369a2eac81f79a71dae3676ce31bf47ae3e4a0d7c782ae296fb5f21b)
BibTeX
@misc{dc40de640b475f,
title = {Verdict Without the Rule: Diagnosing and Auditing Regulatory Rule Sensitivity in LLM Compliance Systems — arXiv:2610.12313},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.12313},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_40de640b475f; sha256 a5611af5369a2eac81f79a71dae3676ce31bf47ae3e4a0d7c782ae296fb5f21b},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.12313. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language model compliance systems are deployed on the assumption that a verdict depends on the regulatory rule it is given. We test this directly across five models and 20 regulatory and platform-policy domains: delete, swap, or negate the governing rule while holding the case fixed, and check whether the verdict changes (OCS) or the model's internal representation of compliance shifts at all (ICS-delta). Neither moves much: models' verdicts are often invariant to substantial perturbations of the supplied rule, and the guard model, evaluated here under a custom-rule adaptation of its native taxonomy, is the least rule-sensitive and least accurate of the five, barely above chance (51%, versus 90-92% for general-purpose models). This reflects easy cases more than blanket neglect: on cases where deleting the rule changes a previously correct model prediction, models do track it closely. Neither better prompting nor direct intervention on the model's internal representations closes this gap. Accuracy alone does not establish that a compliance verdict is grounded in the supplied rule. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Computers and Society (cs.CY). Pagina del lavoro: https://arxiv.org/abs/2610.12313 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T05:47:01.102Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).