Metraipolis › archive › dc_9e431fca9ce2

Beyond Gaussian Worlds: Latent Geometry Matters for JEPAs — arXiv:2609.21656

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.21656
License (as archived)
CC0-1.0
Archived
2026-10-10T01:45:10.049Z
SHA-256 of the archived text
c8a0125a50cfbcc45a9eb2cd30407a9547c04c2b4221441326c7fd278a426f9d
Stable URL of this page
https://metraipolis.com/documento/dc_9e431fca9ce2

How to cite

Beyond Gaussian Worlds: Latent Geometry Matters for JEPAs — arXiv:2609.21656 — arXiv (en), https://arxiv.org/abs/2609.21656 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_9e431fca9ce2 (Metraipolis archive dc_9e431fca9ce2, sha256 c8a0125a50cfbcc45a9eb2cd30407a9547c04c2b4221441326c7fd278a426f9d)
BibTeX
@misc{dc9e431fca9ce2,
  title = {Beyond Gaussian Worlds: Latent Geometry Matters for JEPAs — arXiv:2609.21656},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.21656},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_9e431fca9ce2; sha256 c8a0125a50cfbcc45a9eb2cd30407a9547c04c2b4221441326c7fd278a426f9d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.21656. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Recent Joint-Embedding Predictive Architectures (JEPAs) prevent representation collapse by constraining learned representations to follow a prescribed target distribution, such as an isotropic Gaussian or the uniform distribution on a hypersphere. Klindt et al. (2026) showed that, under their Euclidean assumptions, matching a Gaussian target can recover Gaussian latent variables up to a linear transformation, and that the Gaussian is the unique distribution with this guarantee. We extend their analysis to latent variables supported on embedded Riemannian manifolds and derive conditions on the latent geometry and positive-pair dynamics under which alignment and exact distribution matching guarantee linear recovery. In particular, when the latent variables are uniformly distributed on a sphere and the representations are matched to the same spherical distribution, every optimal representation recovers the latent state up to an orthogonal transformation. This shows that Gaussian uniqueness is not a universal property of distribution-matched JEPAs: non-Euclidean latent geometries can admit other linearly recoverable distributions. We further derive an approximate-recovery bound that is strictly tighter for the spherical world than for the Gaussian world. Experiments on Gaussian, spherical, and toroidal latent spaces show that geometrically compatible targets yield better linear recovery when optimization succeeds, whereas mismatched targets distort the latent structure. This advantage persists in high-dimensional Clifford-torus worlds.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-21 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.21656 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T01:14:16.641Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-21?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).