Metraipolis › archive › dc_fc068c8ea004

Synthetic Hospital: An Open, Verifiable, Physician-Validated Longitudinal EHR Benchmark — arXiv:2609.30027

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.30027
License (as archived): CC0-1.0
Archived: 2026-10-08T04:26:27.131Z
SHA-256 of the archived text: 61404c2e4afaa4ffa9c9a3a4c963371c8e9dea7d492f6b4d73deeca968410e63
Stable URL of this page: https://metraipolis.com/documento/dc_fc068c8ea004

How to cite

Synthetic Hospital: An Open, Verifiable, Physician-Validated Longitudinal EHR Benchmark — arXiv:2609.30027 — arXiv (en), https://arxiv.org/abs/2609.30027 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fc068c8ea004 (Metraipolis archive dc_fc068c8ea004, sha256 61404c2e4afaa4ffa9c9a3a4c963371c8e9dea7d492f6b4d73deeca968410e63)

BibTeX

@misc{dcfc068c8ea004,
  title = {Synthetic Hospital: An Open, Verifiable, Physician-Validated Longitudinal EHR Benchmark — arXiv:2609.30027},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.30027},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fc068c8ea004; sha256 61404c2e4afaa4ffa9c9a3a4c963371c8e9dea7d492f6b4d73deeca968410e63},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.30027. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Frontier language models are rarely used in clinical workflows because the realistic, longitudinal benchmarks needed to develop them are scarce. Real electronic health record (EHR) data cannot be openly shared due to privacy, ethics or data use issues and it does not contain verifiable ground truth since the chart records only reflect what clinicians documented. We introduce Synthetic Hospital, an open, fully synthetic, fact-grounded longitudinal EHR benchmark that resolves the open sharing and verifiable ground truth barriers. Built entirely from public medical-education material with no protected health information, it comprises 1,268 longitudinal patients and 5,602 encounters, where every diagnosis, finding, and temporal relation is grounded in standard ontologies (ICD-10-CM, SNOMED CT, LOINC) and with a complete provenance chain back to its source medical education material. Synthetic Hospital is served through a simulated hospital record system that mirrors real EHR infrastructure (standard interoperability APIs, role-based access and function-calling interface). In a blinded review, physicians distinguished its records from real patient charts at near-chance rates (53\%). Across 10 frontier and open models, none approaches ceiling: the best model reconstructs a patient's longitudinal problem list with a severity-weighted F1 of 0.73, level with the mean of seven physicians on a matched subset but well below the best of them (0.89), and misses roughly half of clinically relevant findings when summarizing a chart. Overall, these results highlight that Synthetic Hospital is a difficult and realistic test of clinical AI performance.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Databases (cs.DB).
Pagina del lavoro: https://arxiv.org/abs/2609.30027 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T03:50:57.861Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).