Metraipolis › archive › dc_710b40f87afe
Counterfactual Model Testing Under Distribution Shift — arXiv:2610.33594
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.33594
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-27T07:56:40.317Z
- SHA-256 of the archived text
3cdb50143085213382b4e32a93548306e41cdef473c659fe80916565c7775286- Stable URL of this page
https://metraipolis.com/documento/dc_710b40f87afe
How to cite
Counterfactual Model Testing Under Distribution Shift — arXiv:2610.33594 — arXiv (en), https://arxiv.org/abs/2610.33594 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_710b40f87afe (Metraipolis archive dc_710b40f87afe, sha256 3cdb50143085213382b4e32a93548306e41cdef473c659fe80916565c7775286)
BibTeX
@misc{dc710b40f87afe,
title = {Counterfactual Model Testing Under Distribution Shift — arXiv:2610.33594},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.33594},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_710b40f87afe; sha256 3cdb50143085213382b4e32a93548306e41cdef473c659fe80916565c7775286},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.33594. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Deployed machine learning (ML) models can degrade as the distribution of their input data shifts. Such shifts can result from changes in the population or from data integrity issues in the pipeline that materializes model features, two sources that operate through different mechanisms. Because labels are often absent or delayed, it is common practice to track shifts in the data instead, but a detected shift does not indicate how it affects the model. We formulate this problem as online model testing under distribution shift. Testing requires reproducing the shift on a controlled baseline with known labels. Since the same shift can come from either source, doing so requires knowing which mechanism changed. We propose a methodology that reproduces observed changes in both mechanisms to estimate current performance and the potential gain from repairing observed data integrity issues. Experiments on real-world datasets with simulated faults show that separating the two sources generally improves both estimates, supporting this formulation. A churn prediction case illustrates how these tests can support monitoring and repair prioritization Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.33594 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-27T07:55:30.647Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).