Metraipolis › archive › dc_6c8d8bfa8490

WA-JEPA: Rethinking the Video JEPA Paradigm for World-Action Modeling in Autonomous Driving — arXiv:2608.20974

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.20974
License (as archived)
CC0-1.0
Archived
2026-10-18T10:53:05.936Z
SHA-256 of the archived text
a3933357c773f3f7c4e0933bc0d01657de2364c733fe7ad043724b96e52c36b2
Stable URL of this page
https://metraipolis.com/documento/dc_6c8d8bfa8490

How to cite

WA-JEPA: Rethinking the Video JEPA Paradigm for World-Action Modeling in Autonomous Driving — arXiv:2608.20974 — arXiv (en), https://arxiv.org/abs/2608.20974 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6c8d8bfa8490 (Metraipolis archive dc_6c8d8bfa8490, sha256 a3933357c773f3f7c4e0933bc0d01657de2364c733fe7ad043724b96e52c36b2)
BibTeX
@misc{dc6c8d8bfa8490,
  title = {WA-JEPA: Rethinking the Video JEPA Paradigm for World-Action Modeling in Autonomous Driving — arXiv:2608.20974},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.20974},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6c8d8bfa8490; sha256 a3933357c773f3f7c4e0933bc0d01657de2364c733fe7ad043724b96e52c36b2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.20974. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Video Joint Embedding Predictive Architecture (V-JEPA) learns powerful spatiotemporal representations from video through self-supervised latent feature prediction. However, V-JEPA is built around random-mask completion and deterministic regression, making it fundamentally ill-suited for autonomous driving planning that demands future-directed prediction tightly coupled with action. To address this, we rethink the V-JEPA paradigm and present WA-JEPA, a V-JEPA-native world-action model designed for autonomous driving planning. Instead of random spatiotemporal masking, WA-JEPA employs hybrid future-masked pre-training, where the model infers future latents from observed context. Departing from deterministic regression, we recast future prediction as conditional flow matching over latent futures, which substantially improves the model's ability to generate plausible future latents for downstream planning. Finally, a joint future-action predictor is proposed to denoise future scene tokens and ego trajectories together in a unified spatiotemporal latent space, allowing action supervision to directly shape planning-relevant world representations. Pre-trained on nuPlan videos and fine-tuned on NAVSIM, WA-JEPA reaches 91.7 EPDMS on NAVSIM-v2, surpassing the strongest end-to-end and world-action baselines by 1.6 and 1.3 EPDMS, and, without HUGSIM-specific fine-tuning, attains the best HD-Score of 0.4462 on the closed-loop HUGSIM benchmark under the same evaluation protocol. These results validate V-JEPA-native world-action modeling as a powerful and scalable paradigm for autonomous driving planning. Code is available at this https URL .

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-24 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.20974 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T10:48:13.585Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).