Metraipolis › archive › dc_ea057a58d4a4

Prior Action Control Causally Alters Language-Model Plasticity — arXiv:2611.02362

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.02362
License (as archived)
CC0-1.0
Archived
2026-11-03T11:22:42.938Z
SHA-256 of the archived text
3f49c1251ee0e8a622dcd7d99aea7aaf00ed14f2c177af5f9198eeb82acce624
Stable URL of this page
https://metraipolis.com/documento/dc_ea057a58d4a4

How to cite

Prior Action Control Causally Alters Language-Model Plasticity — arXiv:2611.02362 — arXiv (en), https://arxiv.org/abs/2611.02362 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ea057a58d4a4 (Metraipolis archive dc_ea057a58d4a4, sha256 3f49c1251ee0e8a622dcd7d99aea7aaf00ed14f2c177af5f9198eeb82acce624)
BibTeX
@misc{dcea057a58d4a4,
  title = {Prior Action Control Causally Alters Language-Model Plasticity — arXiv:2611.02362},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02362},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ea057a58d4a4; sha256 3f49c1251ee0e8a622dcd7d99aea7aaf00ed14f2c177af5f9198eeb82acce624},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02362. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Two copies of one language model start from identical weights and receive the same rewards during training. For one copy the rewards depend on its own answers. The other copy is yoked: it receives the first copy's rewards no matter what it answers, a design taken from the learned-helplessness experiments of animal learning. We ask whether the copy whose rewards depended on its answers, the copy with control, later learns a new task faster, that is, whether control changes the model's plasticity. In Experiment 1, copies of Qwen3-0.6B generated their own answers in a two-turn game and afterwards learned a new task. The copies with control learned the new task faster in every pair, and the advantage held on different starting models as well as on the larger Qwen3-1.7B. Copies with no reward signal, trained only on the updates that every copy shared, learned about as fast as the yoked copies. The advantage is therefore mostly a benefit of control, not damage from lacking it. Control had also changed how the copies answered, so the twins practiced different answers. That practice could carry part of the advantage, and Experiment 2 removed the difference. Both copies of a pair trained on the same answers, fixed in advance, in the same order, and received the same number of rewards. Only one copy's rewards followed the answers, and that copy again learned a new task faster. The advantage of the copies with control over their yoked twins could be forecast before the new task began from each copy's weights. In sum, control over rewards during earlier training left models faster at learning a new task later on, and a conventional model evaluation need not show it.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Neurons and Cognition (q-bio.NC).
Pagina del lavoro: https://arxiv.org/abs/2611.02362 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T11:02:37.607Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).