Metraipolis › archive › dc_4a3ffd56a69b
Learning Under Treatment-Induced Label Indeterminacy with Expert Annotations of Counterfactual Outcomes: A Case Study in Neurological Prognostication — arXiv:2608.12477
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.12477
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T04:27:18.405Z
- SHA-256 of the archived text
5a642b292275d593ae798ee930e34d4f4544ca37dc42b78c87475c857d43818b- Stable URL of this page
https://metraipolis.com/documento/dc_4a3ffd56a69b
How to cite
Learning Under Treatment-Induced Label Indeterminacy with Expert Annotations of Counterfactual Outcomes: A Case Study in Neurological Prognostication — arXiv:2608.12477 — arXiv (en), https://arxiv.org/abs/2608.12477 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4a3ffd56a69b (Metraipolis archive dc_4a3ffd56a69b, sha256 5a642b292275d593ae798ee930e34d4f4544ca37dc42b78c87475c857d43818b)
BibTeX
@misc{dc4a3ffd56a69b,
title = {Learning Under Treatment-Induced Label Indeterminacy with Expert Annotations of Counterfactual Outcomes: A Case Study in Neurological Prognostication — arXiv:2608.12477},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.12477},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4a3ffd56a69b; sha256 5a642b292275d593ae798ee930e34d4f4544ca37dc42b78c87475c857d43818b},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.12477. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Clinical prediction models are often developed as if the outcome of interest were cleanly observed for every patient. This assumption fails when treatment decisions make the clinically relevant outcome permanently unobservable. As a case study of this problem, we consider post-cardiac-arrest neurological prognostication using a cohort of 2,497 patients, including 1,429 patients whose outcomes were rendered indeterminate by treatment decisions. These patients with indeterminate outcomes were reviewed by independent clinical experts, who provided their guesses of counterfactual outcomes about what would have happened to the patients. We refer to these patients as uncertain cases. We also have patients for whom we observe their clinically relevant outcomes; we refer to these patients as certain cases. We propose a framework for evaluating prediction models that explicitly splits the evaluation between certain and uncertain cases. Here, we cannot easily evaluate both types of cases in a uniform manner as the available target labels differ. We then propose a simple prediction model that uses target labels from both certain and uncertain cases in a manner that allows us to trade off between them. Across the proposed neural model and a collection of tabular baselines, models with similar certain-case AUROC can nevertheless differ substantially in both certain-case Brier score and their probability estimates for uncertain cases. Improving alignment with target labels of uncertain cases for our proposed model generally comes at the cost of worse accuracy on certain cases, highlighting an explicit tradeoff that standard evaluation conceals. These results show that when treatment decisions determine whether clinically meaningful outcomes remain observable, conventional evaluation metrics can miss important failure modes in the very patients for whom prognostic support matters most. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-14 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.12477 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T04:20:35.809Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).