Metraipolis › archive › dc_b424c41eb3fb

Talk2Sensors: 3D Visual Grounding in Autonomous Driving via Sensor-Adaptive Physical Cue Matching — arXiv:2608.04568

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.04568
License (as archived)
CC0-1.0
Archived
2026-10-31T11:47:59.352Z
SHA-256 of the archived text
def2524cadb0a4464c2c09e7c3826d573f603c60fa25b1818a14bee488192789
Stable URL of this page
https://metraipolis.com/documento/dc_b424c41eb3fb

How to cite

Talk2Sensors: 3D Visual Grounding in Autonomous Driving via Sensor-Adaptive Physical Cue Matching — arXiv:2608.04568 — arXiv (en), https://arxiv.org/abs/2608.04568 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b424c41eb3fb (Metraipolis archive dc_b424c41eb3fb, sha256 def2524cadb0a4464c2c09e7c3826d573f603c60fa25b1818a14bee488192789)
BibTeX
@misc{dcb424c41eb3fb,
  title = {Talk2Sensors: 3D Visual Grounding in Autonomous Driving via Sensor-Adaptive Physical Cue Matching — arXiv:2608.04568},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.04568},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b424c41eb3fb; sha256 def2524cadb0a4464c2c09e7c3826d573f603c60fa25b1818a14bee488192789},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.04568. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As a key capability for embodied intelligence, 3D visual grounding (3DVG) has been predominantly studied in indoor scenes with RGB-D or point-cloud inputs, while existing outdoor extensions largely rely on monocular images alone. Both settings fall short of real-world outdoor perception, where heterogeneous sensors capture complementary yet distinct physical properties, such as visual texture, 3D geometry, and object kinematics, that are indispensable for flexible and robust query-adaptive grounding but remain under-exploited. To bridge this gap, we introduce Talk2Sensors, the first multi-sensor 3D visual grounding dataset built upon camera, LiDAR, and 4D radar. It contains 8,682 language instructions and 20,558 referred objects, with diverse prompts explicitly aligned with sensor-specific physical cues. Furthermore, we propose TSFormer, a unified Transformer-based framework for language-guided 3D visual grounding in autonomous driving. TSFormer adopts a coarse-to-fine property-aware fusion strategy: the Language-Routed Property Sampler first performs coarse text-conditioned feature retrieval by modulating sensor sampling weights with query-level linguistic cues, while the subsequent Sparse-Preserving Modality Arbiter module conducts fine-grained modality arbitration and text-guided refinement to determine the precise referred spatial location. This design enables dynamic routing of appearance, geometry, and motion cues according to the semantic requirements of each prompt, preventing dense modalities from overwhelming sparse but critical sensor signals. Extensive experiments demonstrate that TSFormer achieves state-of-the-art performance across multiple benchmarks: it improves over the strongest baseline by 8.05 mAP on Talk2Sensors, and transfers to the monocular Mono3DRefer benchmark with 53.05\% [email protected].

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-06 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.04568 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T11:37:40.685Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).