Metraipolis › archive › dc_bf714e6006ba

Training a Conditioned Video Game Agent on a VLM Annotated Dataset — arXiv:2608.05954

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.05954
License (as archived)
CC0-1.0
Archived
2026-10-31T10:37:57.842Z
SHA-256 of the archived text
13ac5bda7883db7a0c985a3b41fe417b55916942a17bd038308514be62e3d6c1
Stable URL of this page
https://metraipolis.com/documento/dc_bf714e6006ba

How to cite

Training a Conditioned Video Game Agent on a VLM Annotated Dataset — arXiv:2608.05954 — arXiv (en), https://arxiv.org/abs/2608.05954 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bf714e6006ba (Metraipolis archive dc_bf714e6006ba, sha256 13ac5bda7883db7a0c985a3b41fe417b55916942a17bd038308514be62e3d6c1)
BibTeX
@misc{dcbf714e6006ba,
  title = {Training a Conditioned Video Game Agent on a VLM Annotated Dataset — arXiv:2608.05954},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.05954},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bf714e6006ba; sha256 13ac5bda7883db7a0c985a3b41fe417b55916942a17bd038308514be62e3d6c1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.05954. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reinforcement Learning (RL) is a powerful but far from easy-to-use technique for policy learning. In the specific case of video games, access to the game engine is required to get rewards for training (e.g. to collect rewards from the environment). Furthermore, the proper identification and weighting of the rewards generally requires a difficult trial-and-error approach. Lastly, rewards are often sparse and understanding how they eventually affect the learned policy is a non-trivial exercise. To ease these issues we propose annotating a video game dataset with Vision Language Models (VLMs) instructed to extract human defined rewards. We show that offline RL can then be used to train a conditioned agent that responds accordingly to the desired returns and we discuss the difficulties and limitations that emerged in our early experiments.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-07 in cs.CV).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.05954 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T10:23:31.088Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).