Metraipolis › archive › dc_dfee645a33b3

When Does Latent Prediction Error Support Physical Surprise Claims in Video World Models? — arXiv:2611.01815

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.01815
License (as archived)
CC0-1.0
Archived
2026-11-03T18:40:35.577Z
SHA-256 of the archived text
ec49bf8018491aab3073c402bf8c74c6b19570a3dacd98a254e0a49c2678d33c
Stable URL of this page
https://metraipolis.com/documento/dc_dfee645a33b3

How to cite

When Does Latent Prediction Error Support Physical Surprise Claims in Video World Models? — arXiv:2611.01815 — arXiv (en), https://arxiv.org/abs/2611.01815 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dfee645a33b3 (Metraipolis archive dc_dfee645a33b3, sha256 ec49bf8018491aab3073c402bf8c74c6b19570a3dacd98a254e0a49c2678d33c)
BibTeX
@misc{dcdfee645a33b3,
  title = {When Does Latent Prediction Error Support Physical Surprise Claims in Video World Models? — arXiv:2611.01815},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01815},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dfee645a33b3; sha256 ec49bf8018491aab3073c402bf8c74c6b19570a3dacd98a254e0a49c2678d33c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01815. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Prediction errors from video world models are often treated as physical surprise, but they can also reflect appearance, static content, motion, or ordinary prediction difficulty. We present an evaluation framework that separates what frozen representations contain, how they organize appearance and physical differences, and what changes a model's own prediction error. On IntPhys2, a benchmark of physically possible and impossible events, we compare representations from latent prediction, masked pixel reconstruction, and simple visual statistics. Appearance remains easy to decode, while V-JEPA2 exposes modest information about whether events are possible or impossible. Looking at the latent geometry, appearance changes become smaller compared with differences between videos, their directions line up more consistently across scenes, and their overlap with physical changes decreases in deeper layers. Across action-conditioned models, incorrect actions raise prediction error, while appearance changes also produce substantial responses. In Physion++ video clips, repeating a single frame can produce higher error than the original clip, while changing color throughout the clip can lower it. Using Kubric, a simulation and rendering framework, we find that impossible events can produce lower error than possible events. A fixed error threshold also incorrectly flags many possible events as impossible. A model can therefore represent physical information and respond to incorrect actions without its native prediction error reliably measuring physical surprise.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.01815 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:12:54.747Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).