Metraipolis › archive › dc_d2a450d511b8
What do VLM-Based Vision-Language Navigation Models Rely on: Interpreting and Steering Policy Behavior — arXiv:2609.24576
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.24576
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T22:06:40.476Z
- SHA-256 of the archived text
79af83fc3a5e7158e740a64c01458349263a260a4375118837fea6cedd3ebd60- Stable URL of this page
https://metraipolis.com/documento/dc_d2a450d511b8
How to cite
What do VLM-Based Vision-Language Navigation Models Rely on: Interpreting and Steering Policy Behavior — arXiv:2609.24576 — arXiv (en), https://arxiv.org/abs/2609.24576 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d2a450d511b8 (Metraipolis archive dc_d2a450d511b8, sha256 79af83fc3a5e7158e740a64c01458349263a260a4375118837fea6cedd3ebd60)
BibTeX
@misc{dcd2a450d511b8,
title = {What do VLM-Based Vision-Language Navigation Models Rely on: Interpreting and Steering Policy Behavior — arXiv:2609.24576},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.24576},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d2a450d511b8; sha256 79af83fc3a5e7158e740a64c01458349263a260a4375118837fea6cedd3ebd60},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.24576. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Modern Vision-Language Navigation (VLN) models rely mostly on pre-trained large Vision-Language Models (VLMs) to predict navigation actions. While this fusion of language instructions and visual observations allows multimodal reasoning, it obscures how information is routed across modalities or what mechanisms drive navigation decisions. Thus, it remains unclear whether VLN models ground their predictions in relevant semantic cues or can track task progress. In this work, we study the interpretability and steerability of VLN models. We use intervention-based metrics that measure how visual observations, instructions, and visual memory causally influence navigation decisions. Our results show that these navigation policies are sensitive to all input modalities and do not depend on a single one. We further show that these agents encode navigation progress and retain semantic structure from their VLM backbones, enabling concept-level steering through internal activations. Finally, we extract activation vectors for abstract behaviors to transfer them zero-shot to out-of-distribution real-world scenarios, improving performance without additional fine-tuning. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CV). Categorie arXiv: Robotics (cs.RO) ; Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.24576 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T21:45:21.154Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).