Metraipolis › archive › dc_37d38586bc0e

Slot-JEPA: Object-Structured Prediction Interfaces for Latent World Models — arXiv:2610.28074

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.28074
License (as archived)
CC0-1.0
Archived
2026-10-21T07:06:59.786Z
SHA-256 of the archived text
6ffb42550bdd31db8842588132ee56084e41a2a211ddf59aac971e335c3191f4
Stable URL of this page
https://metraipolis.com/documento/dc_37d38586bc0e

How to cite

Slot-JEPA: Object-Structured Prediction Interfaces for Latent World Models — arXiv:2610.28074 — arXiv (en), https://arxiv.org/abs/2610.28074 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_37d38586bc0e (Metraipolis archive dc_37d38586bc0e, sha256 6ffb42550bdd31db8842588132ee56084e41a2a211ddf59aac971e335c3191f4)
BibTeX
@misc{dc37d38586bc0e,
  title = {Slot-JEPA: Object-Structured Prediction Interfaces for Latent World Models — arXiv:2610.28074},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.28074},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_37d38586bc0e; sha256 6ffb42550bdd31db8842588132ee56084e41a2a211ddf59aac971e335c3191f4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.28074. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As a prototypical latent world model, Joint-Embedding Predictive Architectures (JEPAs) provide strong semantic representations. However, dense patch representations expose a redundant interface to downstream predictors, which must implicitly infer spatial organization and temporal correspondence, presenting three challenges. First, object information is distributed across patches, forcing each downstream predictor to redundantly recover object structure that could otherwise be reused across tasks before modeling interactions. Second, the lack of explicit entity correspondence across time makes historical transitions expensive, restricting the predictor's flexibility in utilizing context. Third, without explicit transport structure, existing JEPA models typically use pointwise l1/l2 distances, which are insensitive to spatial displacement when predicted and goal supports are largely disjoint. We propose the Slot-based Joint-Embedding Predictive Architecture (Slot-JEPA), which decomposes dense representations into stable semantic slots and time-varying spatial masks. Pretrained once and frozen downstream, this decomposition exposes a reusable object-structured interface. Building on it, our Transition Memory Predictor (TMP) stores visual transitions as compact masks and retrieves useful context using statistics of corresponding slots. Its two branches separate slot-specific dynamics from shared cross-slot dynamics. We further use Unbalanced Optimal Transport (UOT) to supervise mask prediction and evaluate candidate plans by transporting mass between aligned predicted and goal masks, providing a signal better aligned with spatial progress. Across seven visual-understanding benchmarks, the slot-reconstructed features remain transferable. Slot-JEPA also improves goal-conditioned planning across six planar environments and four robotic arm planning tasks on the REALM-JEPA benchmark.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2610.28074 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-21T07:05:43.464Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).