Metraipolis › archive › dc_c46bfebdd7a7

Beyond Raw Engagement: A Counterfactual Observability Framework for Recommender Systems at Netflix — arXiv:2609.22747

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.22747
License (as archived)
CC0-1.0
Archived
2026-10-09T21:58:45.126Z
SHA-256 of the archived text
8a1e658792a7f41028afbab286800b2c2ed36f8505354f2858515116dfd7ecc3
Stable URL of this page
https://metraipolis.com/documento/dc_c46bfebdd7a7

How to cite

Beyond Raw Engagement: A Counterfactual Observability Framework for Recommender Systems at Netflix — arXiv:2609.22747 — arXiv (en), https://arxiv.org/abs/2609.22747 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c46bfebdd7a7 (Metraipolis archive dc_c46bfebdd7a7, sha256 8a1e658792a7f41028afbab286800b2c2ed36f8505354f2858515116dfd7ecc3)
BibTeX
@misc{dcc46bfebdd7a7,
  title = {Beyond Raw Engagement: A Counterfactual Observability Framework for Recommender Systems at Netflix — arXiv:2609.22747},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.22747},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c46bfebdd7a7; sha256 8a1e658792a7f41028afbab286800b2c2ed36f8505354f2858515116dfd7ecc3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22747. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Understanding the performance of large-scale recommender systems remains an underexplored challenge, especially for content creators and model developers. The raw engagement signals available to them, such as views and clicks, conflate content quality, model behavior, presentation bias, and audience reach, making it hard to attribute outcomes to the right cause. In this work, we present a general evaluation framework that enhances observability across multiple recommender systems at Netflix and demonstrate its effectiveness through several production deployments. The framework treats recommender-system observability as a counterfactual measurement problem: estimating what the recommender would have done, and what engagement would have followed, in the absence of a specific content item or model decision. We articulate three stakeholder-centered observability principles for content creators and model developers, and propose measurement methodologies covering bias reduction, relativity, and incrementality, applicable to both single-stage and cascading recommender systems and serving both audiences from a single measurement foundation.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-22 in cs.IR).
Categorie arXiv: Information Retrieval (cs.IR) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.22747 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T21:45:02.203Z dalla pagina https://arxiv.org/catchup/cs.IR/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).