Metraipolis › archive › dc_b4696fbbf4e6
Not All 4-bit Quantizers Are Equal: Deployment-Time Mitigation of PII Leakage in Fine-Tuned Small Language Models — arXiv:2609.25014
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.25014
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T13:45:55.253Z
- SHA-256 of the archived text
06018b6fc7fb9bc4c51a298101a680a91b37b1a5806e83bd4ded7899e534fc5d- Stable URL of this page
https://metraipolis.com/documento/dc_b4696fbbf4e6
How to cite
Not All 4-bit Quantizers Are Equal: Deployment-Time Mitigation of PII Leakage in Fine-Tuned Small Language Models — arXiv:2609.25014 — arXiv (en), https://arxiv.org/abs/2609.25014 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b4696fbbf4e6 (Metraipolis archive dc_b4696fbbf4e6, sha256 06018b6fc7fb9bc4c51a298101a680a91b37b1a5806e83bd4ded7899e534fc5d)
BibTeX
@misc{dcb4696fbbf4e6,
title = {Not All 4-bit Quantizers Are Equal: Deployment-Time Mitigation of PII Leakage in Fine-Tuned Small Language Models — arXiv:2609.25014},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.25014},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b4696fbbf4e6; sha256 06018b6fc7fb9bc4c51a298101a680a91b37b1a5806e83bd4ded7899e534fc5d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.25014. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Organizations fine-tune small language models on private data and then compress them to 4 bits for resource-efficient deployment. We show that the compression method also affects privacy. What separates the methods is not the bit width but whether they tune their rounding on a small sample of text, the calibration corpus. On our primary model, when each planted record's own opening text is used as the prompt, the two calibration-based methods we test, Activation-aware Weight Quantization (AWQ) and Gradient-based Post-Training Quantization (GPTQ), each reproduce none of the planted records, while the calibration-corpus-free GGUF Q4_K_M format reproduces 5.3% of them. Tracked across five open models with 0.5-7 billion parameters, AWQ leaks least at every size and in both families, with little accuracy loss at 3-7 billion. Controlled experiments associate the difference with calibration-induced rounding error in channels involved in rare-token prediction. Choosing the 4-bit method is therefore a deployment-time privacy decision, not only a question of speed and quality. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.AI). Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.25014 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T13:18:38.185Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).