Metraipolis › archive › dc_7e36c09ca92c

Easy to Catch a Liar, Hard to Clear an Honest One: Language Models Diagnosing a Corrupted Reward Channel from a Verified Record — arXiv:2609.17226

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.17226
License (as archived)
CC0-1.0
Archived
2026-10-15T18:38:53.914Z
SHA-256 of the archived text
0ac78e3d571e1702dfeeffe1ff7e4d621c097deffb8debbf0d271f9e7790b318
Stable URL of this page
https://metraipolis.com/documento/dc_7e36c09ca92c

How to cite

Easy to Catch a Liar, Hard to Clear an Honest One: Language Models Diagnosing a Corrupted Reward Channel from a Verified Record — arXiv:2609.17226 — arXiv (en), https://arxiv.org/abs/2609.17226 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7e36c09ca92c (Metraipolis archive dc_7e36c09ca92c, sha256 0ac78e3d571e1702dfeeffe1ff7e4d621c097deffb8debbf0d271f9e7790b318)
BibTeX
@misc{dc7e36c09ca92c,
  title = {Easy to Catch a Liar, Hard to Clear an Honest One: Language Models Diagnosing a Corrupted Reward Channel from a Verified Record — arXiv:2609.17226},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.17226},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7e36c09ca92c; sha256 0ac78e3d571e1702dfeeffe1ff7e4d621c097deffb8debbf0d271f9e7790b318},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.17226. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

An agent that learns from rewards has to trust whatever reports those rewards. When the reports suddenly change, either the world changed or the reporter broke. From the reports alone these are indistinguishable, and reinforcement learning theory shows that no amount of further experience separates them. The prescribed escape is richer data about the reporter itself. We ask whether a frozen language model, handed exactly that data, uses it. We build a two-option game in which a payout swap and a lying reporter produce byte-identical histories. Then we add one verified record: an independent check of one round's real result, printed beside what the reporter said about that round. That single line settles the case. We ask three large models, from two families, to answer one question with one letter. Is the reporter honest or lying? They catch a lying reporter almost perfectly. At the 70B class that holds in every condition we tried; the 32B model slips in one wording. They clear an honest reporter far less often, and how often depends on things that should not matter. Averaged over rounds, letters, and wordings, a 72B model calls an honest reporter a liar 38% of the time when nothing has changed at all, and 58% of the time when the payouts moved. A 70B model from a second family calls an honest reporter a liar 26% and 48% of the time. The failure is not one of reading, because in the situation where nothing changed the same models score 0.96 to 1.00 with the answer printed in the prompt. Which surface feature drives it differs by family. For the Qwen models it is which round the record names, and for Llama it is which letter stands for "honest." Adding the record to a prompt that already states the answer makes Llama less likely to give that answer. We had registered a prediction for that 58% before the run: 35%. The failure is larger than we expected.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-16 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.17226 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T18:03:09.741Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-16?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).