Metraipolis › archive › dc_659ebae69cb4
Does Country Matter? Evaluating Geographic Sensitivity of Large Language Models in Zero-Shot Chronic Kidney Disease Screening — arXiv:2611.08507
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.08507
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-09T22:41:10.388Z
- SHA-256 of the archived text
0e6a563db6efb5529f4e6fd9b00d143fc69d07de2c260a5e6d5aa97eee61585c- Stable URL of this page
https://metraipolis.com/documento/dc_659ebae69cb4
How to cite
Does Country Matter? Evaluating Geographic Sensitivity of Large Language Models in Zero-Shot Chronic Kidney Disease Screening — arXiv:2611.08507 — arXiv (en), https://arxiv.org/abs/2611.08507 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_659ebae69cb4 (Metraipolis archive dc_659ebae69cb4, sha256 0e6a563db6efb5529f4e6fd9b00d143fc69d07de2c260a5e6d5aa97eee61585c)
BibTeX
@misc{dc659ebae69cb4,
title = {Does Country Matter? Evaluating Geographic Sensitivity of Large Language Models in Zero-Shot Chronic Kidney Disease Screening — arXiv:2611.08507},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.08507},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_659ebae69cb4; sha256 0e6a563db6efb5529f4e6fd9b00d143fc69d07de2c260a5e6d5aa97eee61585c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.08507. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) show strong zero-shot reasoning capabilities, yet their sensitivity to prompt context raises questions about reliability in deployment. This study investigates whether geographic context embedded in prompts influences LLM-based chronic kidney disease (CKD) screening, independent of any change in patient features. Using a community-based CKD dataset from Bangladesh (284 participants with 112 CKD and 172 non-CKD), we systematically varied the country attribute across 25 global contexts while holding all clinical inputs constant, and evaluated two open-weight LLMs (Qwen3 8B and Mistral 7B). Experiments were conducted on a common-correct subset of 156 participants for whom both models produced correct baseline predictions. Results show that non-CKD predictions were highly stable across geographic contexts, with both models maintaining consistent non-CKD predictions in nearly all settings. In contrast, CKD-positive predictions were considerably more sensitive. For Qwen3, CKD prediction dropped from a baseline of 65 to as low as 50 across country contexts, while Mistral showed smaller but non-negligible variation. Country contexts did not group according to known epidemiological or healthcare similarities; instead, groupings appear to reflect how country tokens are internally represented within each model. These findings demonstrate that geographic tokens can systematically shift clinical screening outcomes despite identical patient inputs, with implications for the deployment of LLM-based decision support in diverse healthcare settings. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2611.08507 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-09T22:33:47.132Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).