Metraipolis › archive › dc_46ab964bb498
SHq: Learnable Grids for LLM Weight Quantization — arXiv:2611.02966
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.02966
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T18:38:48.773Z
- SHA-256 of the archived text
22dd6922a23930d672a2f45dc79a9bf37acd1db554c243a0ce6e77d50db0fd2d- Stable URL of this page
https://metraipolis.com/documento/dc_46ab964bb498
How to cite
SHq: Learnable Grids for LLM Weight Quantization — arXiv:2611.02966 — arXiv (en), https://arxiv.org/abs/2611.02966 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_46ab964bb498 (Metraipolis archive dc_46ab964bb498, sha256 22dd6922a23930d672a2f45dc79a9bf37acd1db554c243a0ce6e77d50db0fd2d)
BibTeX
@misc{dc46ab964bb498,
title = {SHq: Learnable Grids for LLM Weight Quantization — arXiv:2611.02966},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.02966},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_46ab964bb498; sha256 22dd6922a23930d672a2f45dc79a9bf37acd1db554c243a0ce6e77d50db0fd2d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02966. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Block-scaled lookup-table (LUT) formats, which represent weights as indices into a small value set (the grid), have become standard for low-bit LLM quantization, ranging from NF4 and BOF4 to the 3-bit LUT instruction in NVIDIA Rubin. While these formats leave the grid definition flexible, existing implementations rely on fixed grids derived either from numerical data types (INT, FP) or assumed weight distributions (NF, BOF). We demonstrate that no single fixed grid is universally optimal; the best fixed grid varies with bit-width: BOFS at 2 bits, NF at 4 and 5 bits, and INT at 6 bits. We introduce SHq, a closed-form family of grids parameterized by two critical degrees of freedom: a companding strength {\alpha} that interpolates level spacing from uniform to floating-point-like, and an offset c that introduces asymmetry. This offset allows the sign bit of the FP8 block scale, otherwise unused, to select between the grid and its mirror image, up to doubling the number of distinct values available per block. SHq modifies only the lookup table contents and serves as a drop-in replacement for the grid in any such format. Across five models ranging from 3B to 35B parameters, including a mixture-of-experts architecture, evaluated at 2-6 bits using RTN and GPTQ, SHq achieves the lowest KL divergence in every setting at 3 and 4 bits and in most settings at 2, 5 and 6 bits. It outperforms the best fixed grid by up to 34% and remains superior under mixed-precision configurations. At 4.5 bits per weight, the SHq format reduces the KL divergence of NVFP4 by 41-67% while improving downstream accuracy. Grid families incorporating both spacing and offset parameters perform comparably, whereas those lacking an offset exhibit clearly higher error, confirming that these two degrees of freedom drive the observed gains. We therefore propose SHq as the standard grid for block-scaled lookup-table formats.
Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.02966 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:12:54.604Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).