Metraipolis › archive › dc_25006ea3802c
Efficient Offline Learning of Ranking Policies via Top-$k$ Policy Decomposition — arXiv:2609.36740
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.36740
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T17:29:25.937Z
- SHA-256 of the archived text
a4f3a0380133b744fe601fcd374066ab3bf57d1ddfc4a81b3275daec1fcc8466- Stable URL of this page
https://metraipolis.com/documento/dc_25006ea3802c
How to cite
Efficient Offline Learning of Ranking Policies via Top-$k$ Policy Decomposition — arXiv:2609.36740 — arXiv (en), https://arxiv.org/abs/2609.36740 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_25006ea3802c (Metraipolis archive dc_25006ea3802c, sha256 a4f3a0380133b744fe601fcd374066ab3bf57d1ddfc4a81b3275daec1fcc8466)
BibTeX
@misc{dc25006ea3802c,
title = {Efficient Offline Learning of Ranking Policies via Top-$k$ Policy Decomposition — arXiv:2609.36740},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.36740},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_25006ea3802c; sha256 a4f3a0380133b744fe601fcd374066ab3bf57d1ddfc4a81b3275daec1fcc8466},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36740. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Many recommender systems such as for e-commerce and news platforms aim to provide users with rankings they are likely to interact with. Off-Policy Learning (OPL) of ranking policies enables us to learn new ranking policies using only historical logged data. However, ranking settings make OPL remarkably challenging because their action spaces consist of permutations of unique items, being extremely large. Existing methods primarily use either policy- or regression-based approaches. The policy-based approach, which typically uses importance-weighted policy gradients, can suffer from high variance due to large action spaces. The regression-based approach, on the other hand, estimates the expected reward using conventional machine learning methods, avoiding variance issues but potentially suffering from severe bias. To circumvent these issues of existing methods, we propose a new OPL method for ranking, named Ranking Policy Optimization via Top-$k$ Policy Decomposition (R-POD), which combines the policy- and regression-based approaches in an effective fashion. Specifically, R-POD decomposes a ranking policy into a first-stage policy for selecting top-$k$ actions and a second-stage policy for choosing the bottom actions given the top-$k$ actions. It learns the first-stage policy using a new policy gradient estimator and the second-stage policy via the regression-based approach. This method can substantially reduce variance, since it applies importance weighting only to the top-$k$ actions. We also demonstrate that our policy-gradient estimator for the first-stage policy is unbiased under a conditional pairwise correctness condition, which only requires that the expected reward differences of pairs of rankings sharing the same top-$k$ actions can be estimated correctly. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.36740 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T16:59:48.229Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).