Metraipolis › archive › dc_2c2ba880e8cd
The Effect of Quantization on Clinical Benchmarks: Accuracy and Safety Across Model Families — arXiv:2609.22216
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.22216
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T06:50:24.656Z
- SHA-256 of the archived text
1d38656f362e1eaf060da21b92818760b06dd3be3731d10c9ed9d9b6aad1d5bc- Stable URL of this page
https://metraipolis.com/documento/dc_2c2ba880e8cd
How to cite
The Effect of Quantization on Clinical Benchmarks: Accuracy and Safety Across Model Families — arXiv:2609.22216 — arXiv (en), https://arxiv.org/abs/2609.22216 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2c2ba880e8cd (Metraipolis archive dc_2c2ba880e8cd, sha256 1d38656f362e1eaf060da21b92818760b06dd3be3731d10c9ed9d9b6aad1d5bc)
BibTeX
@misc{dc2c2ba880e8cd,
title = {The Effect of Quantization on Clinical Benchmarks: Accuracy and Safety Across Model Families — arXiv:2609.22216},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.22216},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2c2ba880e8cd; sha256 1d38656f362e1eaf060da21b92818760b06dd3be3731d10c9ed9d9b6aad1d5bc},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22216. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Quantization enables deployment of large language models on resource-constrained clinical edge devices, but its effect on clinical accuracy and safety remains understudied. We evaluate five 7-8B parameter models at FP16, GPTQ-INT8, and GPTQ-INT4 precision across five benchmarks: MedQA, MedMCQA, Med-HALT, a risk-stratified sample of HealthBench, and MedSafetyBench. The study jointly varies quantization bit width, model family, and clinical task type, with explicit risk stratification and safety measures. INT8 GPTQ is universally safe (max. degradation -1.9%-1.9%), while INT4 degradation is substantial and model-dependent: BioMistral-7B, clinically fine-tuned, loses 19.7% on MedMCQA, more than any general-purpose model, showing clinical fine-tuning does not confer compression robustness. MedMCQA degrades more than MedQA under INT4; Med-HALT is largely unaffected. On HealthBench's emergency-risk subgroup, Qwen2.5-7B degrades by 26.8% under INT4, suggesting high-risk scenarios are disproportionately vulnerable to compression. On MedSafetyBench, the model family dominates over precision (refusal rates range 10.2%-74.9% at FP16), though Qwen2.5-7B (-17.8%) and Meditron-7B (-28.3%) show substantial INT4 safety degradation; notably, Qwen2.5-7B is simultaneously the most accuracy-robust model, demonstrating that accuracy and safety robustness are independent properties. We additionally test two recovery methods, clinical calibration substitution and QLoRA fine-tuning, both producing the same trade-off: MedMCQA recovers while MedQA further degrades, indicating recovery strategies require task-specific validation rather than being assumed universally beneficial. These findings indicate INT8 is broadly safe for clinical deployment, while INT4 safety must be assessed per-model and per-task, and that safety alignment is determined primarily by instruction tuning rather than clinical domain adaptation. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CL). Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.22216 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T06:48:45.094Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).