Metraipolis › archive › dc_7943089d53c8

Training seeds and model-selection stability in recommender-system evaluation — arXiv:2609.02499

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.02499
License (as archived)
CC0-1.0
Archived
2026-10-17T05:01:31.537Z
SHA-256 of the archived text
c6ff83d927a7e1323808f81ecc7337383e08ca958ea52039fc23e134bb2c9912
Stable URL of this page
https://metraipolis.com/documento/dc_7943089d53c8

How to cite

Training seeds and model-selection stability in recommender-system evaluation — arXiv:2609.02499 — arXiv (en), https://arxiv.org/abs/2609.02499 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7943089d53c8 (Metraipolis archive dc_7943089d53c8, sha256 c6ff83d927a7e1323808f81ecc7337383e08ca958ea52039fc23e134bb2c9912)
BibTeX
@misc{dc7943089d53c8,
  title = {Training seeds and model-selection stability in recommender-system evaluation — arXiv:2609.02499},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.02499},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7943089d53c8; sha256 c6ff83d927a7e1323808f81ecc7337383e08ca958ea52039fc23e134bb2c9912},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.02499. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Recommender-system experiments often rely on a single random training seed, assuming that run-to-run stochasticity has limited impact on evaluation conclusions. This assumption is risky, as a training seed may influence several algorithm-dependent mechanisms, including parameter initialization, mini-batch ordering, dropout, masking, latent sampling, and training-time negative sampling. We examine this assumption by fixing the data partition and varying the training seed across hyperparameter configurations. We analyze seed effects at three levels: user-level metric sensitivity, validation-based model selection and recommendation-list agreement. Results show that seed variation is often detectable. Its impact depends on whether configurations are clearly separated, whether validation results transfer to test, and whether similar scores lead to similar top-$k$ lists. Findings suggest that reporting single-seed results can overstate the stability of recommender system evaluation, and that training seeds should be treated as part of the evaluation protocol rather than as incidental implementation noise.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-03 in cs.LG).
Categorie arXiv: Information Retrieval (cs.IR) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.02499 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T04:09:43.377Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-03?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).