Metraipolis › archive › dc_ef40e72dd65c

Token-Level Credit Assignment Optimization for Generative Document Retrieval — arXiv:2608.12049

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.12049
License (as archived)
CC0-1.0
Archived
2026-10-19T05:49:02.111Z
SHA-256 of the archived text
cda5994969bbf45b7cb3b6e9e0dff06ec1861a0884dff22f66396e8fd3b2061b
Stable URL of this page
https://metraipolis.com/documento/dc_ef40e72dd65c

How to cite

Token-Level Credit Assignment Optimization for Generative Document Retrieval — arXiv:2608.12049 — arXiv (en), https://arxiv.org/abs/2608.12049 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ef40e72dd65c (Metraipolis archive dc_ef40e72dd65c, sha256 cda5994969bbf45b7cb3b6e9e0dff06ec1861a0884dff22f66396e8fd3b2061b)
BibTeX
@misc{dcef40e72dd65c,
  title = {Token-Level Credit Assignment Optimization for Generative Document Retrieval — arXiv:2608.12049},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.12049},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ef40e72dd65c; sha256 cda5994969bbf45b7cb3b6e9e0dff06ec1861a0884dff22f66396e8fd3b2061b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.12049. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Generative retrieval models perform document retrieval by autoregressively generating document identifiers (DocIDs). This process naturally forms a sequential decision problem, i.e., the model makes a sequence of token-level decisions, selecting a DocID token at each decoding step, with the resulting complete sequence identifying the retrieved document. However, relevance feedback is available only after the complete DocID has been generated and mapped to a document, resulting in a granularity mismatch between token-level generation decisions and document-level retrieval supervision. Consequently, existing reinforcement learning methods for generative retrieval rely on sequence-level rewards, assigning the same document-level relevance signal to every decoding step. Such uniform credit assignment obscures the contribution of individual token decisions, making it difficult to identify which decisions contribute to retrieval success or failure. In this paper, we propose Token-Level Credit Assignment for Generative Retrieval (TCA), a fine-grained reinforcement learning framework that aligns the granularity of credit assignment with that of autoregressive DocID generation. Unlike assigning a single reward to an entire generated DocID, TCA derives fine-grained rewards by comparing the hidden-state trajectory of each generated DocID with the gold DocID trajectory obtained from a frozen reference model. These trajectory-based rewards provide differentiated feedback across decoding steps, allowing the policy to reinforce generation paths that remain aligned with the target DocID. Moreover, TCA decouples token-level credit assignment from policy optimization and can be instantiated with both GRPO and PPO. Experiments on benchmarks show that our method consistently outperforms baselines, demonstrating the effectiveness of fine-grained supervision for aligning DocID generation.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-13 in cs.IR).
Categorie arXiv: Information Retrieval (cs.IR).
Pagina del lavoro: https://arxiv.org/abs/2608.12049 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T05:42:06.597Z dalla pagina https://arxiv.org/catchup/cs.IR/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).