Metraipolis › archive › dc_786c640bae5c
Evolving Semantic Indexes for Generative Retrieval — arXiv:2610.28568
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.28568
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-21T07:07:10.107Z
- SHA-256 of the archived text
5c554389acbeebc92f8db2f3c84355b564c974b6015b80f1afa99811f1e3cb02- Stable URL of this page
https://metraipolis.com/documento/dc_786c640bae5c
How to cite
Evolving Semantic Indexes for Generative Retrieval — arXiv:2610.28568 — arXiv (en), https://arxiv.org/abs/2610.28568 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_786c640bae5c (Metraipolis archive dc_786c640bae5c, sha256 5c554389acbeebc92f8db2f3c84355b564c974b6015b80f1afa99811f1e3cb02)
BibTeX
@misc{dc786c640bae5c,
title = {Evolving Semantic Indexes for Generative Retrieval — arXiv:2610.28568},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.28568},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_786c640bae5c; sha256 5c554389acbeebc92f8db2f3c84355b564c974b6015b80f1afa99811f1e3cb02},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.28568. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Generative retrieval (GR) casts search and recommendation as autoregressive generation of discrete semantic identifiers (SIDs). SID design largely determines GR quality, but its discrete, interdependent components (SID construction, collision handling, decoding, and training) cannot be optimized end-to-end with gradients and remain largely hand-designed. We introduce a framework that improves these components through LLM-driven evolutionary program search, to our knowledge the first application of this approach to GR. A locally served, open-weight 31B model mutates executable GR components, so no proprietary LLM API is used, and a 1M-parameter proxy decoder provides fast feedback in place of repeatedly training the 1B target generator. Across 21 SID programs on NQ320K, proxy and target Recall@100 show strong rank correlation (Spearman $\rho = 0.90$). Guided by this proxy, evolution improves the seed GR on NQ320K, MSMARCO300K, and a multilingual e-commerce dataset under a fixed training recipe. On NQ320K, jointly evolving SID construction and collision handling improves Recall@1 from 55.8% to 59.3% and Recall@100 from 85.8% to 90.6%. A component study shows that proxy reliability depends on what is evolved: improvements transfer reliably for the semantic index but not for the generator's training objective or the full pipeline. LLM-driven evolutionary search is thus a practical tool for GR design, provided proxy reliability is established for the chosen search space. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Information Retrieval (cs.IR); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.28568 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-21T07:05:43.490Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).