Metraipolis › archive › dc_61d4bc474cd2

Feedback That Backfires in Games: Do LLM Players Repeat the Move They Just Watched Fail? — arXiv:2610.30205

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.30205
License (as archived): CC0-1.0
Archived: 2026-10-25T08:31:52.580Z
SHA-256 of the archived text: 5471a2cbbca1ae04f7223626791882136f35d420e7a19c0c479aa9cd84eff5ed
Stable URL of this page: https://metraipolis.com/documento/dc_61d4bc474cd2

How to cite

Feedback That Backfires in Games: Do LLM Players Repeat the Move They Just Watched Fail? — arXiv:2610.30205 — arXiv (en), https://arxiv.org/abs/2610.30205 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_61d4bc474cd2 (Metraipolis archive dc_61d4bc474cd2, sha256 5471a2cbbca1ae04f7223626791882136f35d420e7a19c0c479aa9cd84eff5ed)

BibTeX

@misc{dc61d4bc474cd2,
  title = {Feedback That Backfires in Games: Do LLM Players Repeat the Move They Just Watched Fail? — arXiv:2610.30205},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.30205},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_61d4bc474cd2; sha256 5471a2cbbca1ae04f7223626791882136f35d420e7a19c0c479aa9cd84eff5ed},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.30205. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

When an LLM agent's move fails, the harness writes the move and the failure message into the transcript and asks again. In tool calling, this record makes small models more likely to repeat the failed action. We test whether the same holds in games (Sokoban, Connect Four, Othello), where a move can fail by being illegal or by being legal but losing, both verified exactly. On fixed probe items built by perturbing a correct move, the record backfires: pooled over 8 instruction-tuned models (135M-1.7B) it makes the failed move more likely (corrective gain G = -1.38 nats per token), through the move's surface form, for illegal moves but not for losing ones, and not for every model. An exploratory analysis shows what sets the sign: after the record, the failed move's log-probability barely depends on how likely it was before (slope 0.18). Moves the model would not have played become more likely, and greedy decoding starts to produce them; moves it was about to play become less likely, but greedy decoding still repeats 38% of them. A description of the failure that omits the move string keeps the model's preference instead. Along the OLMo 2 1B post-training ladder the inversion is present in the base model and largely removed at the DPO stage. RAGEN's default models invert in RAGEN's own Sokoban format before any RL. A "do not repeat" instruction does nothing, a clean retry is the worst harness, and a decoder ban stops exact repeats but not illegal play. All measurements ran in float32 on a free Colab GPU, which first reproduced the original paper's number.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.30205 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-25T08:31:12.980Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).