Metraipolis › archive › dc_c4bade9c8b63

UniOMA: Unified Optimal-Transport Multi-Modal Structural Alignment for Robot Perception — arXiv:2611.12351

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.12351
License (as archived)
CC0-1.0
Archived
2026-11-10T18:44:04.346Z
SHA-256 of the archived text
a66dbe417239223afb85eba42c7e5f4b40a87846e0dd54fbb7807c955d0b9a35
Stable URL of this page
https://metraipolis.com/documento/dc_c4bade9c8b63

How to cite

UniOMA: Unified Optimal-Transport Multi-Modal Structural Alignment for Robot Perception — arXiv:2611.12351 — arXiv (en), https://arxiv.org/abs/2611.12351 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c4bade9c8b63 (Metraipolis archive dc_c4bade9c8b63, sha256 a66dbe417239223afb85eba42c7e5f4b40a87846e0dd54fbb7807c955d0b9a35)
BibTeX
@misc{dcc4bade9c8b63,
  title = {UniOMA: Unified Optimal-Transport Multi-Modal Structural Alignment for Robot Perception — arXiv:2611.12351},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.12351},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c4bade9c8b63; sha256 a66dbe417239223afb85eba42c7e5f4b40a87846e0dd54fbb7807c955d0b9a35},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.12351. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Achieving generalizable and well-aligned multimodal representations remains a core challenge in robot learning. While recent approaches have attempted to align modalities by modeling conditional or higher-order statistical dependencies, they do not explicitly consider the structural coherence across modalities. In this work, we propose a novel multimodal alignment method that augments existing contrastive losses with a geometry-aware Gromov--Wasserstein (GW) distance-based regularization. To this end, we encode intra-modal geometry for each modality and reduce cross-modal structural discrepancies through a dynamically learned GW barycenter, thereby encouraging structural consistency in addition to instance-level contrastive alignment. We apply this optimal-transport-based alignment strategy to robot perception tasks involving underexplored modalities such as force and tactile signals, where modality data often exhibit varying sample densities. Extensive experiments on five robotic benchmarks show that our method improves structural alignment and downstream performance over state-of-the-art multimodal contrastive baselines across a wide range of downstream tasks, including robot and environment state prediction. Moreover, our GW-based regularization can be seamlessly integrated into most InfoNCE-like objectives.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Robotics (cs.RO) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.12351 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:37.690Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).