Metraipolis › archive › dc_d514bbdacee6

HerHealthEval: Evaluating Multilingual and Register-Sensitive Understanding of Women's Health Communication — arXiv:2609.20684

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.20684
License (as archived): CC0-1.0
Archived: 2026-10-10T03:48:28.614Z
SHA-256 of the archived text: d7d3aba3628986fe5ced92c97da029cbb3a23e46d5b600345dafc8b891acc3e2
Stable URL of this page: https://metraipolis.com/documento/dc_d514bbdacee6

How to cite

HerHealthEval: Evaluating Multilingual and Register-Sensitive Understanding of Women's Health Communication — arXiv:2609.20684 — arXiv (en), https://arxiv.org/abs/2609.20684 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d514bbdacee6 (Metraipolis archive dc_d514bbdacee6, sha256 d7d3aba3628986fe5ced92c97da029cbb3a23e46d5b600345dafc8b891acc3e2)

BibTeX

@misc{dcd514bbdacee6,
  title = {HerHealthEval: Evaluating Multilingual and Register-Sensitive Understanding of Women's Health Communication — arXiv:2609.20684},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.20684},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d514bbdacee6; sha256 d7d3aba3628986fe5ced92c97da029cbb3a23e46d5b600345dafc8b891acc3e2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.20684. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models are increasingly used in healthcare communication, yet most evaluations emphasize response quality while assuming that the user's concern has been interpreted correctly. We introduce HerHealthEval, a controlled evaluation framework for multilingual understanding of women's-health communication. For each clinical case, HerHealthEval provides matched versions in English, French, and Modern Standard Arabic using six communicative forms: canonical, clinical, layperson, indirect or hedged, emotionally concerned, and deliberately under-specified. The first five express the same underlying concern and retain the same clinical information, whereas the under-specified form intentionally omits relevant details to test whether the model recognizes that clarification is needed. We evaluate a multilingual instruction model and QLoRA-adapted variants on concern classification, risk calibration, clarification behavior, parse compliance, and cross-form consistency. Results reveal that aggregate accuracy and consistency can conceal safety-relevant failures. A multilingual adaptation model reaches 0.994 under-triage in French and Arabic under language-asymmetric risk supervision. A controlled re-adaptation using source-derived, language-invariant risk labels reduces under-triage to 0.572 and 0.558, respectively. These findings show that robust multilingual healthcare evaluation requires explicit testing of register variation, uncertainty handling, and the provenance and invariance of adaptation labels.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-18 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.20684 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T03:24:08.172Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).