Metraipolis › archive › dc_c46bfebdd7a7
Beyond Raw Engagement: A Counterfactual Observability Framework for Recommender Systems at Netflix — arXiv:2609.22747
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.22747
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T21:58:45.126Z
- SHA-256 of the archived text
8a1e658792a7f41028afbab286800b2c2ed36f8505354f2858515116dfd7ecc3- Stable URL of this page
https://metraipolis.com/documento/dc_c46bfebdd7a7
How to cite
Beyond Raw Engagement: A Counterfactual Observability Framework for Recommender Systems at Netflix — arXiv:2609.22747 — arXiv (en), https://arxiv.org/abs/2609.22747 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c46bfebdd7a7 (Metraipolis archive dc_c46bfebdd7a7, sha256 8a1e658792a7f41028afbab286800b2c2ed36f8505354f2858515116dfd7ecc3)
BibTeX
@misc{dcc46bfebdd7a7,
title = {Beyond Raw Engagement: A Counterfactual Observability Framework for Recommender Systems at Netflix — arXiv:2609.22747},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.22747},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c46bfebdd7a7; sha256 8a1e658792a7f41028afbab286800b2c2ed36f8505354f2858515116dfd7ecc3},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22747. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Understanding the performance of large-scale recommender systems remains an underexplored challenge, especially for content creators and model developers. The raw engagement signals available to them, such as views and clicks, conflate content quality, model behavior, presentation bias, and audience reach, making it hard to attribute outcomes to the right cause. In this work, we present a general evaluation framework that enhances observability across multiple recommender systems at Netflix and demonstrate its effectiveness through several production deployments. The framework treats recommender-system observability as a counterfactual measurement problem: estimating what the recommender would have done, and what engagement would have followed, in the absence of a specific content item or model decision. We articulate three stakeholder-centered observability principles for content creators and model developers, and propose measurement methodologies covering bias reduction, relativity, and incrementality, applicable to both single-stage and cascading recommender systems and serving both audiences from a single measurement foundation. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-22 in cs.IR). Categorie arXiv: Information Retrieval (cs.IR) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.22747 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T21:45:02.203Z dalla pagina https://arxiv.org/catchup/cs.IR/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).