Metraipolis › archive › dc_bd0eb441ac8d

Generative Universal Multimodal Retrieval with Dual-role Identifiers — arXiv:2608.12987

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.12987
License (as archived): CC0-1.0
Archived: 2026-10-19T04:18:17.871Z
SHA-256 of the archived text: 7b145ab2a7ba7691c1758144cdff14e3940767d66b8c4837bdd4fe544cff4661
Stable URL of this page: https://metraipolis.com/documento/dc_bd0eb441ac8d

How to cite

Generative Universal Multimodal Retrieval with Dual-role Identifiers — arXiv:2608.12987 — arXiv (en), https://arxiv.org/abs/2608.12987 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bd0eb441ac8d (Metraipolis archive dc_bd0eb441ac8d, sha256 7b145ab2a7ba7691c1758144cdff14e3940767d66b8c4837bdd4fe544cff4661)

BibTeX

@misc{dcbd0eb441ac8d,
  title = {Generative Universal Multimodal Retrieval with Dual-role Identifiers — arXiv:2608.12987},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.12987},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bd0eb441ac8d; sha256 7b145ab2a7ba7691c1758144cdff14e3940767d66b8c4837bdd4fe544cff4661},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.12987. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Generative information retrieval (GIR) has emerged as a compelling alternative to the conventional index-retrieve-then-rank retrieval pipeline by training a generator to produce the identifiers of relevant items directly. Despite its promise, a number of open challenges still remain. First, constrained left-to-right decoding is vulnerable to prefix-level errors and local optima. Second, most prior GIR research remains largely unimodal, leaving instruction-aware retrieval across text, image, and mixed image-text items underexplored. Third, although discrete identifier-based GIR offers higher efficiency, its retrieval accuracy still lags behind that of the cutting-edge dense-vector-based retrieval methods. Motivated by these challenges, we propose DrIG, a novel Generative framework for universal multimodal retrieval featuring Dual-role Identifiers, which supports diverse retrieval tasks across multiple modalities and domains. Each candidate is assigned a single residual-quantized identifier that serves two complementary roles. In its sequential role, the identifier is decoded autoregressively, where the first token explicitly models modality and the remaining tokens capture progressively finer semantics. In its set-based role, the same tokens are reinterpreted as an unordered set to provide a prefix-independent relevance prior, which guides constrained beam search and alleviates local-optimum errors. Extensive experiments on the M-BEIR benchmark and the text-to-image evaluation datasets show that:(1)DrIG consistently outperforms state-of-the-art generative multimodal baselines across diverse tasks, while hybrid reranking achieves a favorable efficiency-effectiveness trade-off against strong dense retrievers. (2)Ablation and scaling analyses reveal how the base LMM, beam size, reranking depth, and fusion strategy affect retrieval performance, providing practical guidance for system design.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-14 in cs.AI).
Categorie arXiv: Information Retrieval (cs.IR) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.12987 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T04:13:06.728Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).