Metraipolis › archive › dc_013eb47f527e
From Weak Data to Strong Policy: Q-Targets Enable Provable In-Context Reinforcement Learning — arXiv:2609.30391
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.30391
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T02:10:20.415Z
- SHA-256 of the archived text
fb1b8a72075ef205f7e55feb9e629455207f230ffed866dd48b5503bb6ea3fd3- Stable URL of this page
https://metraipolis.com/documento/dc_013eb47f527e
How to cite
From Weak Data to Strong Policy: Q-Targets Enable Provable In-Context Reinforcement Learning — arXiv:2609.30391 — arXiv (en), https://arxiv.org/abs/2609.30391 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_013eb47f527e (Metraipolis archive dc_013eb47f527e, sha256 fb1b8a72075ef205f7e55feb9e629455207f230ffed866dd48b5503bb6ea3fd3)
BibTeX
@misc{dc013eb47f527e,
title = {From Weak Data to Strong Policy: Q-Targets Enable Provable In-Context Reinforcement Learning — arXiv:2609.30391},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.30391},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_013eb47f527e; sha256 fb1b8a72075ef205f7e55feb9e629455207f230ffed866dd48b5503bb6ea3fd3},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.30391. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Existing in-context reinforcement learning methods mainly pretrain Transformers with supervised behavior-prediction objectives. This enables task inference from context, but makes the learned policy strongly depend on the quality of offline actions: when trajectories are weak or suboptimal, imitation itself becomes a biased learning signal. We propose Q-Target Pretrained Transformers (QTPT), which keeps the context-conditioned Transformer architecture but replaces behavior cloning with a Bellman-style Q-target objective. QTPT therefore learns to use rewards and transitions in the context to estimate action values, rather than simply imitating the behavior policy. We theoretically analyze QTPT in stochastic linear bandits and finite-horizon MDPs, showing stronger robustness to data quality than supervised pretraining. Empirically, QTPT improves over supervised behavior prediction on controlled RL benchmarks with random or suboptimal data, and we examine extensions to D4RL Kitchen and AntMaze. Supplementary experiments evaluate backbone robustness, meta-RL comparisons, task-coherent context, and unsupported-action value overestimation. These comparisons distinguish the benefits of Q-target pretraining from the remaining limitations of offline coverage. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.30391 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T01:43:06.658Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).