Metraipolis › archive › dc_7cb8c0b3df41

Institution-Specific LLM Prompting Recovers PHI That De-identification Systems and Their Gold Standards Both Miss — arXiv:2608.17051

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.17051
License (as archived)
CC0-1.0
Archived
2026-10-18T14:29:08.935Z
SHA-256 of the archived text
a3b92990385af4f5b406a15477c21ec23610904074aadf275c13c25981e4bfd8
Stable URL of this page
https://metraipolis.com/documento/dc_7cb8c0b3df41

How to cite

Institution-Specific LLM Prompting Recovers PHI That De-identification Systems and Their Gold Standards Both Miss — arXiv:2608.17051 — arXiv (en), https://arxiv.org/abs/2608.17051 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7cb8c0b3df41 (Metraipolis archive dc_7cb8c0b3df41, sha256 a3b92990385af4f5b406a15477c21ec23610904074aadf275c13c25981e4bfd8)
BibTeX
@misc{dc7cb8c0b3df41,
  title = {Institution-Specific LLM Prompting Recovers PHI That De-identification Systems and Their Gold Standards Both Miss — arXiv:2608.17051},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.17051},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7cb8c0b3df41; sha256 a3b92990385af4f5b406a15477c21ec23610904074aadf275c13c25981e4bfd8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.17051. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Secondary use of electronic health records requires de-identification, yet existing systems miss \emph{institutionally situated} protected health information (PHI) such as hospital abbreviations, building names, and internal codes whose status is locally determined. We ask whether large language models (LLMs) with in-context learning (ICL) can close this gap and control the precision--recall trade-off. On 100 annotated pediatric oncology notes (5,322 PHI spans) from Texas Children's Hospital, we benchmarked eight LLMs against two purpose-built systems (Stanford TiDE, OpenMed PII) and two pattern-based baselines. Each LLM ran under three prompts of increasing specificity: (1) a HIPAA-aligned baseline, (2) baseline plus the institutional PHI categories it missed, and (3) prompt 2 plus instructions against over-redacting clinical content. We then compared 14~multi-agent and ensemble configurations against the best single prompt, with recall the primary safety metric. LLMs outperformed the purpose-built systems (best F1=0.918$\pm$0.001 vs.\ TiDE 0.779), with advantages concentrated in contextual categories. Naming the missed categories recovered 79\% (48/61) of them, and discouraging over-redaction restored precision. No agentic architecture beat calibrated single-pass prompting (F1 0.906--0.907), but LLM outputs surfaced 414~candidate annotation gaps; re-annotation confirmed 227~PHI spans, against which the final prompt reached recall=0.981 (F1=0.907$\pm$0.002). Well-calibrated ICL resolves both the institutional PHI gap and the precision--recall trade-off in one LLM call per note. LLMs cost more to run than traditional methods, but that cost buys a way to audit the reference standard. LLMs are a legitimate, adaptable alternative to purpose-built de-identification systems; institution-specific prompt development should be the primary adaptation strategy.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-19 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.17051 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T14:27:54.367Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-19?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).