Metraipolis › archive › dc_e68528599151
Unimodal Augmentation-based SSL Objectives Induce Cross-Modal Alignment — arXiv:2611.02917
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.02917
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T11:23:00.477Z
- SHA-256 of the archived text
345f2346013efac46d7f2ddf8c90abd788013745523e193066de5a8c4581de04- Stable URL of this page
https://metraipolis.com/documento/dc_e68528599151
How to cite
Unimodal Augmentation-based SSL Objectives Induce Cross-Modal Alignment — arXiv:2611.02917 — arXiv (en), https://arxiv.org/abs/2611.02917 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e68528599151 (Metraipolis archive dc_e68528599151, sha256 345f2346013efac46d7f2ddf8c90abd788013745523e193066de5a8c4581de04)
BibTeX
@misc{dce68528599151,
title = {Unimodal Augmentation-based SSL Objectives Induce Cross-Modal Alignment — arXiv:2611.02917},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.02917},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e68528599151; sha256 345f2346013efac46d7f2ddf8c90abd788013745523e193066de5a8c4581de04},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02917. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Recent works have observed implicit cross-modal alignment between independently trained unimodal models. However, the mechanisms underlying the emergence of this phenomenon remain unresolved. In this paper, we demonstrate that unimodal augmentation-based self-supervised objectives implicitly induce cross-modal alignment. We show that low unimodal losses force high cross-modal alignment, whereas a high unimodal loss prevents such alignment. We theoretically connect the widely used InfoNCE loss to local CKA, the popular measure of cross-modal alignment. We further provide extensive empirical evidence supporting this claim, both in controlled settings and across 53 unconstrained frozen text, vision, and audio unimodal encoders. In both cases, unimodal augmentation-based objectives control cross-modal alignment. Overall, our results provide a theoretical and empirical foundation for understanding the emergence of implicit cross-modal alignment by establishing a direct connection between unimodal training objectives and cross-modal structure. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Machine Learning (stat.ML). Pagina del lavoro: https://arxiv.org/abs/2611.02917 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T11:02:37.656Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).