Metraipolis › archive › dc_7a532eccd205

SHq: Learnable Grids for LLM Weight Quantization — arXiv:2611.02966

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.02966
License (as archived)
CC0-1.0
Archived
2026-11-03T11:23:01.763Z
SHA-256 of the archived text
867085f68ab9cabf5e892a86173e20b9093f451e4d55005d84ef1becfcce0523
Stable URL of this page
https://metraipolis.com/documento/dc_7a532eccd205

How to cite

SHq: Learnable Grids for LLM Weight Quantization — arXiv:2611.02966 — arXiv (en), https://arxiv.org/abs/2611.02966 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7a532eccd205 (Metraipolis archive dc_7a532eccd205, sha256 867085f68ab9cabf5e892a86173e20b9093f451e4d55005d84ef1becfcce0523)
BibTeX
@misc{dc7a532eccd205,
  title = {SHq: Learnable Grids for LLM Weight Quantization — arXiv:2611.02966},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02966},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7a532eccd205; sha256 867085f68ab9cabf5e892a86173e20b9093f451e4d55005d84ef1becfcce0523},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02966. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Block-scaled lookup-table (LUT) formats, which represent weights as indices into a small value set (the grid), have become standard for low-bit LLM quantization, ranging from NF4 and BOF4 to the 3-bit LUT instruction in NVIDIA Rubin. While these formats leave the grid definition flexible, existing implementations rely on fixed grids derived either from numerical data types (INT, FP) or assumed weight distributions (NF, BOF). We demonstrate that no single fixed grid is universally optimal; the best fixed grid varies with bit-width: BOFS at 2 bits, NF at 4 and 5 bits, and INT at 6 bits. We introduce SHq, a closed-form family of grids parameterized by two critical degrees of freedom: a companding strength {\alpha} that interpolates level spacing from uniform to floating-point-like, and an offset c that introduces asymmetry. This offset allows the sign bit of the FP8 block scale, otherwise unused, to select between the grid and its mirror image, up to doubling the number of distinct values available per block. SHq modifies only the lookup table contents and serves as a drop-in replacement for the grid in any such format. Across five models ranging from 3B to 35B parameters, including a mixture-of-experts architecture, evaluated at 2-6 bits using RTN and GPTQ, SHq achieves the lowest KL divergence in every setting at 3 and 4 bits and in most settings at 2, 5 and 6 bits. It outperforms the best fixed grid by up to 34% and remains superior under mixed-precision configurations. At 4.5 bits per weight, the SHq format reduces the KL divergence of NVFP4 by 41-67% while improving downstream accuracy. Grid families incorporating both spacing and offset parameters perform comparably, whereas those lacking an offset exhibit clearly higher error, confirming that these two degrees of freedom drive the observed gains. We therefore propose SHq as the standard grid for block-scaled lookup-table formats.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2611.02966 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T11:02:37.660Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).