Metraipolis › archive › dc_3008a21c56bb
"Dear LLaVA, Please Drive": A Depth-Aware Vision-Language Agent for Closed-Loop Robotic Control — arXiv:2609.22925
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.22925
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T22:23:41.957Z
- SHA-256 of the archived text
f18bfb5c72e43c3d371c4851229875f543cb6bf9b68db1ff0e7d0c7b042921b7- Stable URL of this page
https://metraipolis.com/documento/dc_3008a21c56bb
How to cite
"Dear LLaVA, Please Drive": A Depth-Aware Vision-Language Agent for Closed-Loop Robotic Control — arXiv:2609.22925 — arXiv (en), https://arxiv.org/abs/2609.22925 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3008a21c56bb (Metraipolis archive dc_3008a21c56bb, sha256 f18bfb5c72e43c3d371c4851229875f543cb6bf9b68db1ff0e7d0c7b042921b7)
BibTeX
@misc{dc3008a21c56bb,
title = {"Dear LLaVA, Please Drive": A Depth-Aware Vision-Language Agent for Closed-Loop Robotic Control — arXiv:2609.22925},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.22925},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3008a21c56bb; sha256 f18bfb5c72e43c3d371c4851229875f543cb6bf9b68db1ff0e7d0c7b042921b7},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22925. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Vision-language models (VLMs) provide a compelling foundation for reasoning-driven mobile navigation, offering rich contextual understanding and strong generalization from large-scale pretraining. Most existing navigation frameworks rely on imitation learning and therefore require substantial labeled trajectory data, limiting their scalability and robustness. In this work, we propose a parameter-efficient approach to fine-tune a pretrained VLM for autonomous navigation using an Imperative Learning paradigm. By optimizing against differentiable geometric cost fields rather than labeled trajectories, our model learns to generate collision-free paths exclusively from stereoscopic depth observations. We introduce a unified end-to-end navigation pipeline for natural-language-driven robotic control. This system leverages a shared VLM backbone with task-specific Low-Rank Adaptation (LoRA) modules, effectively bridging the gap from semantic target selection to low-level trajectory planning. Our approach achieves competitive Success weighted by Path Length (SPL) in unseen environments while updating less than 1% of the model's total parameters. Qualitative real-world experiments validate sim-to-real generalization and stable path planning without fine-tuning on real-world data. These results highlight a practical approach for deploying VLM-based agents on mobile robots, enabling high-level semantic navigation without the prohibitive requirement for large-scale, labeled trajectory data. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-22 in cs.RO). Categorie arXiv: Robotics (cs.RO). Pagina del lavoro: https://arxiv.org/abs/2609.22925 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T21:59:59.504Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).