Metraipolis › archive › dc_fa901bbac258

Unimodal Augmentation-based SSL Objectives Induce Cross-Modal Alignment — arXiv:2611.02917

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.02917
License (as archived): CC0-1.0
Archived: 2026-11-03T18:38:46.563Z
SHA-256 of the archived text: 763b955a3778503cf3e1e1df7b3443f2b0df26bb349a96684d73c20d42034def
Stable URL of this page: https://metraipolis.com/documento/dc_fa901bbac258

How to cite

Unimodal Augmentation-based SSL Objectives Induce Cross-Modal Alignment — arXiv:2611.02917 — arXiv (en), https://arxiv.org/abs/2611.02917 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fa901bbac258 (Metraipolis archive dc_fa901bbac258, sha256 763b955a3778503cf3e1e1df7b3443f2b0df26bb349a96684d73c20d42034def)

BibTeX

@misc{dcfa901bbac258,
  title = {Unimodal Augmentation-based SSL Objectives Induce Cross-Modal Alignment — arXiv:2611.02917},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02917},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fa901bbac258; sha256 763b955a3778503cf3e1e1df7b3443f2b0df26bb349a96684d73c20d42034def},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02917. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Recent works have observed implicit cross-modal alignment between independently trained unimodal models. However, the mechanisms underlying the emergence of this phenomenon remain unresolved. In this paper, we demonstrate that unimodal augmentation-based self-supervised objectives implicitly induce cross-modal alignment. We show that low unimodal losses force high cross-modal alignment, whereas a high unimodal loss prevents such alignment. We theoretically connect the widely used InfoNCE loss to local CKA, the popular measure of cross-modal alignment. We further provide extensive empirical evidence supporting this claim, both in controlled settings and across 53 unconstrained frozen text, vision, and audio unimodal encoders. In both cases, unimodal augmentation-based objectives control cross-modal alignment. Overall, our results provide a theoretical and empirical foundation for understanding the emergence of implicit cross-modal alignment by establishing a direct connection between unimodal training objectives and cross-modal structure.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2611.02917 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:12:54.601Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).