Metraipolis › archive › dc_bb688e10ef9e
Sim-to-Real Traffic Scene Understanding by Decoupling Semantics from Caption Generation with V-JEPA — arXiv:2609.18562
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.18562
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T16:16:37.032Z
- SHA-256 of the archived text
8adeb241018a82f3b88511b9e8edb6e8fc44146aa0b1e1db5df9d645d2a57ef0- Stable URL of this page
https://metraipolis.com/documento/dc_bb688e10ef9e
How to cite
Sim-to-Real Traffic Scene Understanding by Decoupling Semantics from Caption Generation with V-JEPA — arXiv:2609.18562 — arXiv (en), https://arxiv.org/abs/2609.18562 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bb688e10ef9e (Metraipolis archive dc_bb688e10ef9e, sha256 8adeb241018a82f3b88511b9e8edb6e8fc44146aa0b1e1db5df9d645d2a57ef0)
BibTeX
@misc{dcbb688e10ef9e,
title = {Sim-to-Real Traffic Scene Understanding by Decoupling Semantics from Caption Generation with V-JEPA — arXiv:2609.18562},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.18562},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bb688e10ef9e; sha256 8adeb241018a82f3b88511b9e8edb6e8fc44146aa0b1e1db5df9d645d2a57ef0},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.18562. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Track 2 of the AI City Challenge 2026 requires both visual question answering (VQA) and traffic event description generation under a challenging synthetic-to real domain shift. Existing vision-language approaches often entangle semantic understanding with language generation, making them susceptible to hallucination and inconsistent reasoning across event phases. In this work, we propose a decoupled semantic understanding framework that first resolves predefined traffic questions into structured semantic facts and subsequently uses these facts to guide caption generation. A frozen V-JEPA encoder extracts predictive scene representations, while a lightweight Llama-based predictor produces answers for VQA queries. To improve reliability, we introduce a training-free structured refinement mechanism that exploits statistical priors, inter-question relationships, and temporal event consistency to correct prediction errors. The refined semantic facts are then provided to Qwen3-VL-8B to generate pedestrian and vehicle descriptions for each traffic event. Experimental results on the official 2026 AI City Challenge Track 2 benchmark show that the proposed method achieves 87.09% VQA accuracy and an overall S2 score of 60.0853, ranking first among all participating teams. These results demonstrate that predictive world representations combined with structured semantic refinement enable more accurate and reliable traffic understanding, leading to higher-quality lan guage generation. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-17 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.18562 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T15:45:19.806Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).