Metraipolis › archive › dc_12fea96ebfa5

VTInstructor: Visual Trajectory Prompting for Navigation Instruction Generation in Continuous Environments — arXiv:2608.15284

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.15284
License (as archived): CC0-1.0
Archived: 2026-10-18T18:02:19.678Z
SHA-256 of the archived text: d8e31799226035474e48bf8c95023285f0d170639405ba15615133403eac494d
Stable URL of this page: https://metraipolis.com/documento/dc_12fea96ebfa5

How to cite

VTInstructor: Visual Trajectory Prompting for Navigation Instruction Generation in Continuous Environments — arXiv:2608.15284 — arXiv (en), https://arxiv.org/abs/2608.15284 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_12fea96ebfa5 (Metraipolis archive dc_12fea96ebfa5, sha256 d8e31799226035474e48bf8c95023285f0d170639405ba15615133403eac494d)

BibTeX

@misc{dc12fea96ebfa5,
  title = {VTInstructor: Visual Trajectory Prompting for Navigation Instruction Generation in Continuous Environments — arXiv:2608.15284},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.15284},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_12fea96ebfa5; sha256 d8e31799226035474e48bf8c95023285f0d170639405ba15615133403eac494d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.15284. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Navigation instruction generation from ego-centric RGB video in continuous environments is an important yet challenging task for human-robot interaction and scalable dataset construction. Prior instruction generators assume discrete viewpoint graphs with panoramic observations, where trajectory structure is explicit; in continuous environments, however, the agent receives only a dense RGB stream, making trajectory cues difficult to recover. We propose VTInstructor, the first VLN instruction generation framework for continuous environments. Our key idea is to convert implicit trajectory geometry into explicit visual trajectory prompts: EDTC condenses long RGB trajectories into navigation-critical keyframes, VTP overlays path, turn, and goal cues onto these anchors, VTMod injects the resulting trajectory signals into the visual encoder, and VT-GRPO further calibrates this spatial injection during training, all without requiring a navigation graph, pre-built map, or scene reconstruction. On the challenging R2R-CE and RxR-CE Val Unseen benchmarks, VTInstructor sets a new state of the art across all standard NLG metrics, surpassing the strongest baseline by +0.357 CIDEr and +0.109 CIDEr, respectively. Beyond automatic metrics, VTInstructor-generated instructions raise a frozen follower's success rate to 63.3%, a +14.7 percentage-point gain over the best competing instruction source, and provide consistent data augmentation gains of +3 SR points on downstream navigation tasks.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI).
Categorie arXiv: Robotics (cs.RO) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM).
Pagina del lavoro: https://arxiv.org/abs/2608.15284 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T16:29:14.002Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).