Metraipolis › archive › dc_430c06ee7547

Can Frontier LLMs Match Natively Multimodal Embeddings? A Comparison on Hard-Negative Text-to-Image Retrieval — arXiv:2608.11343

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.11343
License (as archived)
CC0-1.0
Archived
2026-10-19T05:15:19.620Z
SHA-256 of the archived text
161f33f1a05e499812b88b281a344430cabebba59902e2a7f7a213d0e28c8f63
Stable URL of this page
https://metraipolis.com/documento/dc_430c06ee7547

How to cite

Can Frontier LLMs Match Natively Multimodal Embeddings? A Comparison on Hard-Negative Text-to-Image Retrieval — arXiv:2608.11343 — arXiv (en), https://arxiv.org/abs/2608.11343 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_430c06ee7547 (Metraipolis archive dc_430c06ee7547, sha256 161f33f1a05e499812b88b281a344430cabebba59902e2a7f7a213d0e28c8f63)
BibTeX
@misc{dc430c06ee7547,
  title = {Can Frontier LLMs Match Natively Multimodal Embeddings? A Comparison on Hard-Negative Text-to-Image Retrieval — arXiv:2608.11343},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.11343},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_430c06ee7547; sha256 161f33f1a05e499812b88b281a344430cabebba59902e2a7f7a213d0e28c8f63},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11343. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multimodal retrieval and classification across different types of media, spanning text, images,video and audio, has traditionally relied on dual-encoder models that align visual and textual representations through contrastive learning. The March 2026 release of Gemini Embedding 2, Google's first natively multimodal embedding model to map text, images, video, audio, and documents into a single shared space, raises competition among multimodal retrieval systems. Simultaneously, frontier Large language models (LLMs) have also demonstrated strong visual understanding, raising the question of whether they can serve as effective zero-shot rankers. Our study provides the first direct comparison of native multimodal embeddings against LLM-based visual ranking on Flickr30k. We observe that GPT-4.1 and Claude Sonnet 4.6 perform on par with Gemini Embedding 2. Additionally, once embeddings are precomputed, multimodal embeddings are better suited for low-latency applications.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-13 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computer Vision and Pattern Recognition (cs.CV); Information Retrieval (cs.IR); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.11343 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T05:08:46.620Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).