Metraipolis › archive › dc_acbc00d32df2

EHRAdapt: Adapting Pretrained Language Models to Electronic Health Records with Semantic Priors for Rare Clinical Events — arXiv:2609.34007

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.34007
License (as archived): CC0-1.0
Archived: 2026-10-07T20:57:17.506Z
SHA-256 of the archived text: 3cde2b312ae2141fe2c3650a858fd9cbbd319f2ee18f8c4dd33d799475eb19e7
Stable URL of this page: https://metraipolis.com/documento/dc_acbc00d32df2

How to cite

EHRAdapt: Adapting Pretrained Language Models to Electronic Health Records with Semantic Priors for Rare Clinical Events — arXiv:2609.34007 — arXiv (en), https://arxiv.org/abs/2609.34007 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_acbc00d32df2 (Metraipolis archive dc_acbc00d32df2, sha256 3cde2b312ae2141fe2c3650a858fd9cbbd319f2ee18f8c4dd33d799475eb19e7)

BibTeX

@misc{dcacbc00d32df2,
  title = {EHRAdapt: Adapting Pretrained Language Models to Electronic Health Records with Semantic Priors for Rare Clinical Events — arXiv:2609.34007},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.34007},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_acbc00d32df2; sha256 3cde2b312ae2141fe2c3650a858fd9cbbd319f2ee18f8c4dd33d799475eb19e7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34007. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Electronic health records (EHRs) encode clinical histories as (time, modality, code) tuples, whereas pretrained language models expect text tokens. Serializing them as text inflates sequence length and redundantly encodes structure. We introduce EHRAdapt, an adapter that maps tuples directly into a frozen language model's embedding space. Modality receives a learned embedding, time gaps enter through learned attention biases, and event codes receive dedicated vectors. Learning event vectors is the central challenge: clinical vocabularies are long-tailed, leaving rare events too few observations for reliable estimates. EHRAdapt therefore represents each event vector as the sum of a semantic prior and an evidence residual. The prior is a frozen embedding of the event's clinical description from a biomedical language model trained on clinical ontologies, mapped into the model's input space by a shared learned projection, so it supplies clinical meaning even when observations are scarce. The residual, a learned low-rank event-specific correction, refines it as evidence accumulates. We run continued pretraining on about 4 million patients' records with three frozen LLM backbones (OLMo2 1B, Llama3.2 1B, and OLMo2 7B), training only the adapter (0.1--0.6% of all parameters). The full adapter outperforms all ablations in held-out next-event prediction on every backbone. Removing the semantic pathway hurts rare events over ten times more than the most frequent ones, whereas removing the residual hurts overall prediction but improves it for the rarest events. On reportable infectious-disease and syndromic downstream classification tasks, EHRAdapt outperforms text-based LLM and count-based baselines, and both pathways improve rare-disease discrimination. The two pathways therefore play complementary roles, visible only when results are broken down by event frequency rather than averaged.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.34007 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:01.401Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).