Metraipolis › archive › dc_996845e2ac35
Omni-Interactive Universal Embedder — arXiv:2608.27044
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.27044
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T14:23:22.287Z
- SHA-256 of the archived text
a6e1ba00c8add469517bbcbc7037f29d184e825b57647cf2afa7754e172a8d4d- Stable URL of this page
https://metraipolis.com/documento/dc_996845e2ac35
How to cite
Omni-Interactive Universal Embedder — arXiv:2608.27044 — arXiv (en), https://arxiv.org/abs/2608.27044 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_996845e2ac35 (Metraipolis archive dc_996845e2ac35, sha256 a6e1ba00c8add469517bbcbc7037f29d184e825b57647cf2afa7754e172a8d4d)
BibTeX
@misc{dc996845e2ac35,
title = {Omni-Interactive Universal Embedder — arXiv:2608.27044},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.27044},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_996845e2ac35; sha256 a6e1ba00c8add469517bbcbc7037f29d184e825b57647cf2afa7754e172a8d4d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.27044. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Multimodal representation learning has been shifting from traditional two-tower architectures to large language model (LLM)-based embedders due to their strong instruction-following capabilities. Despite this progress, existing approaches primarily focus on language and image modalities, which also remain the dominant modalities for user-conditioned interactions in current embedders. In this paper, we propose the first Omni-Interactive Universal Embedder (OmniUE), which not only learns a unified embedding space across text, video, and audio by leveraging intermediate-layer representations from dedicated learnable tokens, but also supports omni-interactive querying, enabling users to provide inputs in the form of text, visual regions of interest, and audio spans. Within OmniUE, visual and audio segmenters process diverse user interactions and integrate them with an omni-LLM to produce user-conditioned any-to-any embeddings via context aggregation. To evaluate OmniUE's omni-interactive capabilities, we introduce OmniCHOIR, benchmarking models for omni-interactive compositional audio retrieval based on the given text, video, and audio as well as unimodal or multimodal interaction prompts. OmniUE consistently surpasses state-of-the-art baselines across diverse modalities, with average improvements of 10.5% on textual-interactive video benchmarks (MMEB-v2-video), 1.1% on audio tasks (MAEB), 83.7% on visual-interactive benchmarks (SCaR), and 24.1% on our omni-interactive OmniCHOIR benchmark. We believe that jointly advancing omni-modal representation learning and omni-interactive querying paves the way toward universal embedders. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-28 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2608.27044 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T13:16:53.542Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).