Metraipolis › archive › dc_b5e274f3bf07
GateTrim: Adaptive-Rank Quantization for Large Language Models — arXiv:2611.00654
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.00654
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T11:20:42.685Z
- SHA-256 of the archived text
a1a740d3d30a795e8c8d6824a866f9c0e4b1962d5da48df290506ccdf29fbf1f- Stable URL of this page
https://metraipolis.com/documento/dc_b5e274f3bf07
How to cite
GateTrim: Adaptive-Rank Quantization for Large Language Models — arXiv:2611.00654 — arXiv (en), https://arxiv.org/abs/2611.00654 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b5e274f3bf07 (Metraipolis archive dc_b5e274f3bf07, sha256 a1a740d3d30a795e8c8d6824a866f9c0e4b1962d5da48df290506ccdf29fbf1f)
BibTeX
@misc{dcb5e274f3bf07,
title = {GateTrim: Adaptive-Rank Quantization for Large Language Models — arXiv:2611.00654},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.00654},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b5e274f3bf07; sha256 a1a740d3d30a795e8c8d6824a866f9c0e4b1962d5da48df290506ccdf29fbf1f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00654. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We introduce GateTrim, an adaptive-rank post-training quantization method for large language models that learns matrix-specific ranks through gate optimization and threshold-based rank selection. GateTrim standardizes the low-rank factors of a quantized low-rank--residual representation, learns a separate gate vector for each target weight matrix, and uses a shared threshold to automatically determine its retained rank. The resulting ranks can differ across matrices, and the compact quantized representations are subsequently reconstructed. We evaluate GateTrim under W4A8 on Qwen3-4B/8B/14B, Llama-3.2-3B-Instruct, and Llama-3.1-8B-Instruct. Across all five models, we observe a counterintuitive finding: increasing the threshold from $\tau=0.3$ to $\tau=0.7$ reduces the mean retained rank by 46.60%--67.89% while simultaneously lowering WikiText-2 perplexity by 0.59%--5.17%. Threshold 0.7 achieves the lowest mean retained rank and final perplexity among the three tested thresholds, 0.3, 0.5, and 0.7, and reduces measured target-export size by 1.50%--4.25% relative to 0.3. GateTrim uses an $8\times512$-token fitting pool and a separate development pool of the same size. Its Qwen3-4B/8B/14B perplexities at $\tau=0.7$ are numerically below TwinQuant's published W4A8 values, reported with a $128\times2048$-token calibration pool, in a cross-study comparison. On Qwen3-8B, the threshold-0.7 configuration already has the lowest perplexity before reconstruction under both tested initialization seeds; reconstruction improves it further without increasing retained rank or export size. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2611.00654 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T11:02:37.463Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).