Metraipolis › archive › dc_331f8f75534e

When Can Self-Play Fix a False Belief? Answer Keys, Pass Bands and Replacement Rules in an Execution-Grounded Loop — arXiv:2611.06965

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.06965
License (as archived)
CC0-1.0
Archived
2026-11-06T14:51:26.160Z
SHA-256 of the archived text
f71982dce3eee98a21f130387a18f888dadab5cc057bc3cfd99a8fe135d96fa8
Stable URL of this page
https://metraipolis.com/documento/dc_331f8f75534e

How to cite

When Can Self-Play Fix a False Belief? Answer Keys, Pass Bands and Replacement Rules in an Execution-Grounded Loop — arXiv:2611.06965 — arXiv (en), https://arxiv.org/abs/2611.06965 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_331f8f75534e (Metraipolis archive dc_331f8f75534e, sha256 f71982dce3eee98a21f130387a18f888dadab5cc057bc3cfd99a8fe135d96fa8)
BibTeX
@misc{dc331f8f75534e,
  title = {When Can Self-Play Fix a False Belief? Answer Keys, Pass Bands and Replacement Rules in an Execution-Grounded Loop — arXiv:2611.06965},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.06965},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_331f8f75534e; sha256 f71982dce3eee98a21f130387a18f888dadab5cc057bc3cfd99a8fe135d96fa8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.06965. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Self-play loops in which a model writes tasks, derives an answer key and trains on its own matching solutions are often called grounded when the key comes from executing code. We ask which false beliefs such a loop removes. We run expert-iteration self-play with Qwen3-4B-Base on output-prediction tasks, vary the source of the answer key, and track natural beliefs about Python semantics and two misreadings of task specifications planted by fine-tuning ("at most n" read as < n; "the k-th smallest" read as 0-indexed). Executing the generator's own code gave a key that encoded the "at most" misreading on every audited task, and training on it never raised the truth rate above 0.03. A key from an implementation by the unplanted model, a teacher without the error, raised it to 0.23-0.58 after three rounds in four runs. A correct key is not sufficient: the pass band, which keeps tasks solved 1-7 times of 8, admits almost no corrective task while a belief is held on nearly every sample. Execution keys sharply reduced both natural runtime beliefs on diagnostic items in both seeds. Corrections came with replacement rules that fail on neighbouring items the old belief got right. The robust case, present in all four spec runs, is that after the "at most" fix the model increasingly counts the nearest value above a limit that is absent from the list (overcount 0.18-0.38 of control samples at round 3), a case our planted generators almost never wrote (4 of 10,924 tasks); diagnostic-only metrics miss it. Asked for those cases, the planted generator wrote none in 2,061 checkable tasks, against 27% for the untrained unplanted model in one round. Both generators often moved the request into the specification instead ("the cap itself does not count"), and a decorrelated key follows the specification.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.SE, nuovi lavori del giorno).
Categorie arXiv: Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2611.06965 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-06T14:21:57.528Z dalla pagina https://arxiv.org/list/cs.SE/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).