Metraipolis › archive › dc_4e610a5d92af
Do-JEPA: From Masking to Intervention in Latent World Models — arXiv:2609.37378
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.37378
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T16:27:13.387Z
- SHA-256 of the archived text
6fba7569bb25fd0cd9035ed5fd8fb00f07d007381f8d75b2f1f632a1068e06f9- Stable URL of this page
https://metraipolis.com/documento/dc_4e610a5d92af
How to cite
Do-JEPA: From Masking to Intervention in Latent World Models — arXiv:2609.37378 — arXiv (en), https://arxiv.org/abs/2609.37378 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4e610a5d92af (Metraipolis archive dc_4e610a5d92af, sha256 6fba7569bb25fd0cd9035ed5fd8fb00f07d007381f8d75b2f1f632a1068e06f9)
BibTeX
@misc{dc4e610a5d92af,
title = {Do-JEPA: From Masking to Intervention in Latent World Models — arXiv:2609.37378},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.37378},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4e610a5d92af; sha256 6fba7569bb25fd0cd9035ed5fd8fb00f07d007381f8d75b2f1f632a1068e06f9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.37378. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Latent world models are trained to predict what happens next, so nothing in their objective separates what an action caused from what merely co-occurred with it. Object-masking models such as C-JEPA intervene on what the predictor can see; we intervene on what physically happens. From one saved simulator state we run the dynamics under an action $a$ and under a reference action $a_{\varnothing}$, and train the model to predict the difference $\Delta z=z^{a}-z^{a_{\varnothing}}$ between the two latent futures. The resulting objective, Do-JEPA, has an effect loss, a support loss (where the action enters), a propagation loss (where its effect travels) and invariance losses (what must not change). In a synthetic system with object-aligned variables, support supervision finds the directly intervened object in 99.95% of test cases, where a sparse action mask sends the action to a nuisance slot in every case, and response-onset supervision recovers the ring-shaped propagation graph (edge AUROC 0.975 vs. 0.624). From pixels, the effect loss beats a control trained on exactly the same data: it lowers latent effect error by 28.4% on an end-to-end LeWM model and physical effect error by 13.5% when trained and tested on natural action sequences, and on three independently generated CausalWorld benchmarks it lowers responsive effect error by about 20% under physics shifts and the latent context sensitivity of predicted effects by 66%. Trained from scratch it costs factual accuracy; fine-tuning an existing model with it removes this cost. Together, these results show that intervening on the world, rather than on what the model sees, helps latent world models predict what their actions cause.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.37378 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T15:54:53.977Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).