Metraipolis › archive › dc_89660a2d7827

Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation — arXiv:2610.02148

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.02148
License (as archived)
CC0-1.0
Archived
2026-10-06T19:15:37.746Z
SHA-256 of the archived text
65f2a94f8997a7c97ec4e40d8588526d6b49bb9d34797c19dcbd1067db765080
Stable URL of this page
https://metraipolis.com/documento/dc_89660a2d7827

How to cite

Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation — arXiv:2610.02148 — arXiv (en), https://arxiv.org/abs/2610.02148 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_89660a2d7827 (Metraipolis archive dc_89660a2d7827, sha256 65f2a94f8997a7c97ec4e40d8588526d6b49bb9d34797c19dcbd1067db765080)
BibTeX
@misc{dc89660a2d7827,
  title = {Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation — arXiv:2610.02148},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.02148},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_89660a2d7827; sha256 65f2a94f8997a7c97ec4e40d8588526d6b49bb9d34797c19dcbd1067db765080},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.02148. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Extending a text embedding model to new modalities typically degrades text retrieval quality, and existing omni-modal embedders compensate with multi-billion parameters. We present Omni-Embed-Mini, a 0.9B-parameter model that maps text, speech, audio, images, video, and visually-rich documents into a single shared cosine space without updating any text-side parameter. Our key insight is that the teacher signal requires no separate embedding model: each media sample is paired with a dense cascaded caption, and the teacher target is simply the frozen backbone's own embedding of that caption. Because teacher and student share the same backbone weights, they inhabit byte-identical geometry, and lightweight projectors plus phased LoRA adapters on the modality encoders suffice for alignment. Training combines a Matryoshka SigLIP contrastive loss with an online hybrid hard-negative miner whose negatives sharpen as the encoder improves. The recipe carries over to a 2.3B variant by swapping in a native vision-language backbone. Omni-Embed-Mini-0.9B keeps its text weights bit-identical to the backbone, so training cannot regress text retrieval (49.57 nDCG@10 on MTEB-v2 BEIR-8), while extending it to five additional modalities, and is ~2.7x to 9.5x smaller than every open omni embedder we compare against. The 2.3B variant is competitive with the closed gemini-embedding-2, edging ahead of it on the overall-modality average. Models, code, data and evaluation harness are on our project page: this https URL

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-10-02 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2610.02148 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T17:37:36.887Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).