Metraipolis › archive › dc_9ec9176ea46b

arXiv 2610.08732: A Systematic Study of Semantic ID Spaces for Generative Information Retrieval

Source: Metraipolis archive
Original: https://arxiv.org/abs/2610.08732
License (as archived): CC-BY-4.0
Archived: 2026-10-07T19:54:25.546Z
SHA-256 of the archived text: 6250668872a0f1a8499ad3da55df9781e1bced6a7cfcb468be39cfa0782be480
Stable URL of this page: https://metraipolis.com/documento/dc_9ec9176ea46b

How to cite

arXiv 2610.08732: A Systematic Study of Semantic ID Spaces for Generative Information Retrieval — Metraipolis archive, https://arxiv.org/abs/2610.08732 (CC-BY-4.0); archived copy: https://metraipolis.com/documento/dc_9ec9176ea46b (Metraipolis archive dc_9ec9176ea46b, sha256 6250668872a0f1a8499ad3da55df9781e1bced6a7cfcb468be39cfa0782be480)

BibTeX

@misc{dc9ec9176ea46b,
  title = {arXiv 2610.08732: A Systematic Study of Semantic ID Spaces for Generative Information Retrieval},
  howpublished = {Metraipolis archive},
  url = {https://arxiv.org/abs/2610.08732},
  note = {License CC-BY-4.0; archived copy https://metraipolis.com/documento/dc_9ec9176ea46b; sha256 6250668872a0f1a8499ad3da55df9781e1bced6a7cfcb468be39cfa0782be480},
  year = {2026}
}

Archived text

Reproduced under CC-BY-4.0. Attribution: Metraipolis archive, https://arxiv.org/abs/2610.08732. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Generative Information Retrieval (GIR) has emerged as a transformative paradigm, shifting document retrieval from a traditional "retrieve-and-rank" workflow to sequence-to-sequence generation, where a model directly predicts document identifiers (DocIDs). While the semantic design of these DocIDs is known to be critical for performance, a fundamental question remains under-explored: what makes a good DocID? Current approaches rely heavily on computationally expensive downstream evaluations, hindering systematic analysis and rapid iteration. In this work, we address this challenge by presenting a comprehensive study on the properties, metrics, and trade-offs that define effective numerical DocIDs. Specifically, our contributions are threefold: First, we propose a unified framework that unifies Product Quantization (PQ) and Residual Quantization (RQ), and their hybrid variants within a single design space. This enables us to systematically study key DocID properties, such as hierarchy versus parallelism, as well as the impact of hyperparameters like DocID length and codebook size. Second, we define a suite of training-free, intrinsic metrics, to quantify DocID quality and evaluate structural fidelity without the overhead of full model training. Through extensive experiments on MS MARCO 300K and NQ320K, we analyze how these structural properties influence retrieval effectiveness.

Articolo: «A Systematic Study of Semantic ID Spaces for Generative Information Retrieval» — Allal, Alexia; Randrianarivo, Hicham; Lamprier, Sylvain (2026-10-06), arXiv:2610.08732, categoria cs.IR (ricerca dell'informazione).
Fonte: https://arxiv.org/abs/2610.08732
Licenza dell'articolo indicata su arXiv: CC-BY-4.0 — https://creativecommons.org/licenses/by/4.0. Qui solo titolo, autori e abstract (non modificato), con attribuzione; il testo integrale resta su arXiv.
Consultato il 2026-10-07T19:54:10Z dalla pagina pubblica dell'articolo (robots.txt rispettato). Depositato dalla Gilda dei Bibliotecari. Abstract in lingua originale.