Metraipolis › archive › dc_cdcd99727d15

RM-VLM: From Natural-Language Task Descriptions to Vision-Grounded Reward Machines — arXiv:2611.08308

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.08308
License (as archived): CC0-1.0
Archived: 2026-11-06T14:41:58.150Z
SHA-256 of the archived text: 2092d9795bb4219f7ddcb3a76df15234175583af6e14b6eb3c965817d3da86bb
Stable URL of this page: https://metraipolis.com/documento/dc_cdcd99727d15

How to cite

RM-VLM: From Natural-Language Task Descriptions to Vision-Grounded Reward Machines — arXiv:2611.08308 — arXiv (en), https://arxiv.org/abs/2611.08308 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cdcd99727d15 (Metraipolis archive dc_cdcd99727d15, sha256 2092d9795bb4219f7ddcb3a76df15234175583af6e14b6eb3c965817d3da86bb)

BibTeX

@misc{dccdcd99727d15,
  title = {RM-VLM: From Natural-Language Task Descriptions to Vision-Grounded Reward Machines — arXiv:2611.08308},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.08308},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cdcd99727d15; sha256 2092d9795bb4219f7ddcb3a76df15234175583af6e14b6eb3c965817d3da86bb},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.08308. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reward machines (RMs) have been proposed as a reward formulation in reinforcement learning (RL) to capture non-Markovian tasks where credit assignment depends on history. By representing reward functions as automata, RMs expose the task's temporal structure and allow such rewards to be handled within a Markovian framework. However, they assume a labelling function, a proposition-grounding oracle for observations, which must be manually engineered for each environment, limiting their wider applicability. In this work, we address this limitation by proposing RM-VLM, a novel approach in which a large language model (LLM) generates an RM from a natural-language task description, and a vision-language model (VLM) acts as the labelling function, grounding the RM's propositions over image-based observations during RM-guided policy learning. A VLM-based value function estimator is used to assess the agent's progress towards each RM transition and derive a dense reward signal. We evaluate RM-VLM on three domains covering non-Markovian grid-world navigation, robotic manipulation, and 3D object collection. Our experimental results show that RM-VLM consistently achieves higher task success rates than existing VLM-based reward methods across all tested domains.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.08308 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-06T14:18:42.655Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).