Metraipolis › archive › dc_56ace26049eb
When Do Surrogate Metrics Work? A Finite-Sample Comparison Under Realistic Failure Modes — arXiv:2609.30528
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.30528
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T02:49:58.185Z
- SHA-256 of the archived text
46d43e11168bf137d67d5dee89cde15cd13e61e70d0ac461e34c6c61c396e449- Stable URL of this page
https://metraipolis.com/documento/dc_56ace26049eb
How to cite
When Do Surrogate Metrics Work? A Finite-Sample Comparison Under Realistic Failure Modes — arXiv:2609.30528 — arXiv (en), https://arxiv.org/abs/2609.30528 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_56ace26049eb (Metraipolis archive dc_56ace26049eb, sha256 46d43e11168bf137d67d5dee89cde15cd13e61e70d0ac461e34c6c61c396e449)
BibTeX
@misc{dc56ace26049eb,
title = {When Do Surrogate Metrics Work? A Finite-Sample Comparison Under Realistic Failure Modes — arXiv:2609.30528},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.30528},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_56ace26049eb; sha256 46d43e11168bf137d67d5dee89cde15cd13e61e70d0ac461e34c6c61c396e449},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.30528. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Online experiments must often be evaluated before long-term outcomes mature. Under rolling enrollment, these outcomes are observed only for early enrollees, while short-term surrogates are available for everyone. We compare seven estimators across eleven data-generating processes, spanning partial mediation, drift, outcome sparsity, and enrollment-time labeling, with up to $R = 2,000$ replications over more than 500 method-by-scenario cells. We find a sharp robustness-efficiency tradeoff: the surrogate index delivers large efficiency gains when surrogacy holds but its coverage collapses under violations, while PPI-family methods stay asymptotically valid under random labeling at smaller gains. We give the finite-sample variance of PPI++ in the all-units parameterization for a fixed predictor, a joint asymptotic distribution for the two estimators under cross-fitting, and a Hausman-type estimator-disagreement diagnostic, then quantify the detection-damage gap: in the partial-mediation design, where we locate both edges, a band of violations destroys surrogate-index coverage yet is too small to detect on most datasets. On the 64,000-customer Hillstrom experiment the diagnostic rarely flags a violation that biases the surrogate index, and a Cauchy-kernel hybrid of the two estimators inherits 10.7% relative bias; on the 14-million-user Criteo experiment the violation is detected, and subsampling traces detection turning on with scale as damage persists. PPI++ has limits: at a rare-conversion $n = 30,000$ Criteo subsample its empirical coverage is 85.0%. We recommend prespecifying PPI++ with the exact variance as the primary analysis under random labeling and adequate labeled outcome counts, reading the diagnostic as a warning, not a certificate, and treating the surrogate index as a sensitivity analysis. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-28 in stat.ME). Categorie arXiv: Methodology (stat.ME) ; Econometrics (econ.EM); Applications (stat.AP). Pagina del lavoro: https://arxiv.org/abs/2609.30528 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T02:13:17.806Z dalla pagina https://arxiv.org/catchup/stat.ME/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).