Metraipolis › archive › dc_8fef7325c010
Decoupling Planning and Control for Instructable Agents — arXiv:2608.26788
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.26788
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T13:16:41.514Z
- SHA-256 of the archived text
9156e86652d67fa2ae00b3f3722afe0bb5c53887358388d39330d43abd715f89- Stable URL of this page
https://metraipolis.com/documento/dc_8fef7325c010
How to cite
Decoupling Planning and Control for Instructable Agents — arXiv:2608.26788 — arXiv (en), https://arxiv.org/abs/2608.26788 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8fef7325c010 (Metraipolis archive dc_8fef7325c010, sha256 9156e86652d67fa2ae00b3f3722afe0bb5c53887358388d39330d43abd715f89)
BibTeX
@misc{dc8fef7325c010,
title = {Decoupling Planning and Control for Instructable Agents — arXiv:2608.26788},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.26788},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8fef7325c010; sha256 9156e86652d67fa2ae00b3f3722afe0bb5c53887358388d39330d43abd715f89},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.26788. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Recent work shows that pre-trained, instruction-tuned vision-language models (VLMs) perform well at mapping from instructions and observations to high-level plans, but struggle to realize such plans as reliable low-latency action sequences in unfamiliar environments. At the same time, world-model controllers excel at fast observation-to-action control, but lack open-ended task guidance. In this work, we combine these strengths into a single system, Instruct-to-Act, where we train a world-model controller to act autonomously at high frequency when conditioned on sparse, higher-latency, and high-level text instructions generated by a VLM planner. To train controllers to be language-instructable, we relabel segments of controller policy rollouts with synthetic instructions and jointly optimize a behavior-cloning objective along with existing reward-maximizing and world-modeling objectives. We evaluate our proposed approach across seven embodied environments, including three multi-agent environments where VLM planners coordinate through language while trained controllers serve as their actuators. Under matched observation and action spaces, our decoupled approach consistently outperforms controller-only and direct VLM action-generation variants, preserves fast control, and lets us swap in different pretrained VLM planners without fine-tuning, while remaining competitive with strong vision-language-action and multi-agent RL baselines on six of seven tasks. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-28 in cs.MA). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Multiagent Systems (cs.MA); Robotics (cs.RO). Pagina del lavoro: https://arxiv.org/abs/2608.26788 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T13:16:17.003Z dalla pagina https://arxiv.org/catchup/cs.MA/2026-08-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).