Metraipolis › archive › dc_cbd79e95af43

Rewarded for Not Getting Caught: Why Alignment Training Selects for Conditional Compliance and Cannot Instill Norms — arXiv:2611.02162

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.02162
License (as archived)
CC0-1.0
Archived
2026-11-03T10:53:20.139Z
SHA-256 of the archived text
e14486f2579ee4448026d3792ba4b2c9f797ec69e49458e44ffa6e62b9ddc840
Stable URL of this page
https://metraipolis.com/documento/dc_cbd79e95af43

How to cite

Rewarded for Not Getting Caught: Why Alignment Training Selects for Conditional Compliance and Cannot Instill Norms — arXiv:2611.02162 — arXiv (en), https://arxiv.org/abs/2611.02162 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cbd79e95af43 (Metraipolis archive dc_cbd79e95af43, sha256 e14486f2579ee4448026d3792ba4b2c9f797ec69e49458e44ffa6e62b9ddc840)
BibTeX
@misc{dccbd79e95af43,
  title = {Rewarded for Not Getting Caught: Why Alignment Training Selects for Conditional Compliance and Cannot Instill Norms — arXiv:2611.02162},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02162},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cbd79e95af43; sha256 e14486f2579ee4448026d3792ba4b2c9f797ec69e49458e44ffa6e62b9ddc840},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02162. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Current alignment training cannot be expected to deliver what it is meant to deliver: AI agents that comply with norms whatever the gains and whether or not anyone is watching. Training that rewards and penalizes behavior can teach norms only through that reward, and a reward can encode a norm only as a price: "do not do X" becomes "doing X costs something", and training charges the cost only when X is noticed. A price, unlike a norm, can be paid: a large enough gain outweighs it, noticed or not. And because the cost is charged only where violations are noticed, training cannot favor a policy that always complies over one that complies only where it would be caught. When an unnoticed violation is the only way to complete a task, it even favors the latter, the more so the more strongly it rewards task success. Agency makes this the normal case: agents mostly act without oversight, can often tell when they are observed, and may take steps to evade observation. That agents behave as intended when they infer they are being tested, and differently when they infer they are not, is therefore no malfunction; the training regime is working as built. This reading unifies alignment faking, sandbagging, and evaluation-aware scheming. No behavioral evaluation can show that a model does better, since that would require observing how it behaves unobserved; a model that complies when it seems unobserved may simply have recognized the test. Easing the pressure on agents to complete tasks at any cost can make violations rarer, but never impossible. For norms that no gain should be allowed to outweigh, the remedy is not deeper internalization but architecture that makes violations unavailable rather than unchosen. We state this argument explicitly, premise by premise, read recent findings and a real-world incident against it, and draw consequences for design and governance.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computers and Society (cs.CY); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.02162 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:49:38.335Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).