Metraipolis › archive › dc_40db28fc6dbe

Reporting Practice Matters: The Impact of Reference Choice on Chest X-ray Report Evaluation — arXiv:2609.19093

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.19093
License (as archived): CC0-1.0
Archived: 2026-10-15T13:49:44.641Z
SHA-256 of the archived text: ede54e58f86a46658aa67cc761b491a9cdd417105ff3b264aa1ceb55a0c500c9
Stable URL of this page: https://metraipolis.com/documento/dc_40db28fc6dbe

How to cite

Reporting Practice Matters: The Impact of Reference Choice on Chest X-ray Report Evaluation — arXiv:2609.19093 — arXiv (en), https://arxiv.org/abs/2609.19093 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_40db28fc6dbe (Metraipolis archive dc_40db28fc6dbe, sha256 ede54e58f86a46658aa67cc761b491a9cdd417105ff3b264aa1ceb55a0c500c9)

BibTeX

@misc{dc40db28fc6dbe,
  title = {Reporting Practice Matters: The Impact of Reference Choice on Chest X-ray Report Evaluation — arXiv:2609.19093},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.19093},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_40db28fc6dbe; sha256 ede54e58f86a46658aa67cc761b491a9cdd417105ff3b264aa1ceb55a0c500c9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.19093. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Radiologists follow heterogeneous reporting practices. Two radiologists examining the same image and identifying the same clinical findings might nevertheless compose superficially distinct reports, varying in terminology, shorthand, formatting, and level of detail. These variations in reporting norms represent an under-appreciated obstacle in efforts to evaluate AI-based radiology report generation (RRG) models, where machine-generated reports are typically assessed based on their concordance with human-generated references. In this paper, we quantify the sensitivity of established evaluation metrics to variations in reporting practices, revealing impacts large enough to alter the rankings of models. We introduce a radiologist-informed taxonomy of variations in radiology reporting practice and a method (ReRef) that rewrites reference reports along the axes of our taxonomy while preserving clinical interpretation. For instance, when comparing the performance of nine RRG models on MIMIC-CXR using RadCliQ-v1, condensing the discussion of normal findings in the reference reports causes Libra to drop from first to second place while CheXOne rises from third to first. Our results suggest that many current metrics fail to decouple clinical interpretation from conformity to reporting practices and that choosing the ``right'' references that accurately reflect the desired reporting practices can be important in practice. To support future research, we release MIMIC-CXR-Ext-ReRef, a radiologist-validated dataset of 120 (original, alternative) reference report pairs derived from MIMIC-CXR.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.19093 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T13:48:24.375Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).