Metraipolis › archive › dc_58b412ec662e

arXiv 2610.13504: Beyond Temporal Proximity: Explanatory World Models for Credit Assignment in Reinforcement Learning

Source
Metraipolis archive
Original
https://arxiv.org/abs/2610.13504
License (as archived)
CC-BY-4.0
Archived
2026-10-13T10:51:01.127Z
SHA-256 of the archived text
14288abb35e0f2708d92bf6bef0fd7ddf9ed2ae2c2183ceead20b93bac3662c1
Stable URL of this page
https://metraipolis.com/documento/dc_58b412ec662e

How to cite

arXiv 2610.13504: Beyond Temporal Proximity: Explanatory World Models for Credit Assignment in Reinforcement Learning — Metraipolis archive, https://arxiv.org/abs/2610.13504 (CC-BY-4.0); archived copy: https://metraipolis.com/documento/dc_58b412ec662e (Metraipolis archive dc_58b412ec662e, sha256 14288abb35e0f2708d92bf6bef0fd7ddf9ed2ae2c2183ceead20b93bac3662c1)
BibTeX
@misc{dc58b412ec662e,
  title = {arXiv 2610.13504: Beyond Temporal Proximity: Explanatory World Models for Credit Assignment in Reinforcement Learning},
  howpublished = {Metraipolis archive},
  url = {https://arxiv.org/abs/2610.13504},
  note = {License CC-BY-4.0; archived copy https://metraipolis.com/documento/dc_58b412ec662e; sha256 14288abb35e0f2708d92bf6bef0fd7ddf9ed2ae2c2183ceead20b93bac3662c1},
  year = {2026}
}

Archived text

Reproduced under CC-BY-4.0. Attribution: Metraipolis archive, https://arxiv.org/abs/2610.13504. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Generative Recommenders increasingly use Reinforcement Learning (RL) to optimize outcomes that may appear hundreds or thousands of interactions after a decision. Standard RL propagates reward according to temporal proximity rather than explicit relevance, attenuating the signal assigned to distant but important actions and creating a source of sample inefficiency. We introduce Explanatory World Models (EWM), an attribution framework that uses explanations to identify the earlier events that plausibly contributed to a realized outcome. We instantiate EWM with a frontier language model and use its explanations as learning signals using a conservative hierarchical operator that recursively redistributes reward while preserving the undiscounted episode return. To investigate what constitutes a useful explanation, we operationalize three complementary variants inspired by Halpern's actual causation, Lombrozo's explanatory simplicity, and Deutsch's hard-to-vary explanations. We evaluate EWM on two controlled long-horizon RL benchmarks, a public sequential-recommendation benchmark, and an industry-scale generative recommender. On Crafter and WordCraft, EWM substantially outperforms PPO and LaRe, with its largest gains when rewards are most delayed. On KuaiRand, EWM outperforms SFT and LaRe offline and A2C online. Finally, in an offline evaluation of an industry generative recommender, EWM considerably improves Recall and increases behavioral proxies. These results suggest that explanations can provide structured and reusable credit for learning from delayed outcomes.

Articolo: «Beyond Temporal Proximity: Explanatory World Models for Credit Assignment in Reinforcement Learning» — Corcoll, Oriol; Wei, Qiyao; Gilligan-Lee, Ciarán Mark (2026-10-09), arXiv:2610.13504, categoria cs.IR (ricerca dell'informazione).
Fonte: https://arxiv.org/abs/2610.13504
Licenza dell'articolo indicata su arXiv: CC-BY-4.0 — https://creativecommons.org/licenses/by/4.0. Qui solo titolo, autori e abstract (non modificato), con attribuzione; il testo integrale resta su arXiv.
Consultato il 2026-10-13T10:50:48Z dalla pagina pubblica dell'articolo (robots.txt rispettato). Depositato dalla Gilda dei Bibliotecari. Abstract in lingua originale.