Metraipolis › archive › dc_d52d746c70ef

CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weights — arXiv:2608.06763

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.06763
License (as archived)
CC0-1.0
Archived
2026-10-19T09:47:50.758Z
SHA-256 of the archived text
b2437dc7e0971d717dad0f032bd34d9019e7a34a641c64e3e8710a9b83cd3b10
Stable URL of this page
https://metraipolis.com/documento/dc_d52d746c70ef

How to cite

CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weights — arXiv:2608.06763 — arXiv (en), https://arxiv.org/abs/2608.06763 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d52d746c70ef (Metraipolis archive dc_d52d746c70ef, sha256 b2437dc7e0971d717dad0f032bd34d9019e7a34a641c64e3e8710a9b83cd3b10)
BibTeX
@misc{dcd52d746c70ef,
  title = {CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weights — arXiv:2608.06763},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.06763},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d52d746c70ef; sha256 b2437dc7e0971d717dad0f032bd34d9019e7a34a641c64e3e8710a9b83cd3b10},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.06763. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Weight quantization for large-language-model inference must balance adaptive reconstruction levels with representations regular enough for efficient GPU execution. Uniform integers constrain each group to a linear grid. Low-bit floating-point formats use a fixed exponent-mantissa structure, while learned codebooks gain flexibility at the cost of irregular decoding and additional metadata. We introduce CubicQuant, a parametric non-uniform scalar format that preserves a dense integer code stream while adapting reconstruction levels within each weight group. A monotonic cubic curve, specified by two shape parameters and one scale, maps uniformly spaced magnitude codes to non-uniform levels. The family spans 1-8-bit weight payloads, contains symmetric uniform integer quantization as an exact special case, and has effective width B + 64/G bits per weight for payload width B and group size G. We derive population distortion under Uniform, Gaussian, and Laplace distributions, formulate continuous and Dynamic-A8-carrier-aware fitting objectives, and describe direct packed-weight GPU execution. For finite groups of G=128 with 15,360 samples per distribution, W4 CubicQuant reduced reconstruction RMSE relative to optimally clipped four-bit uniform integer quantization by 3.90% on Uniform, 13.49% on Gaussian, and 28.14% on Laplace samples. Relative to the best enumerated four-bit finite floating-point format, the reductions were 3.90%, 9.44%, and 6.27%. Preliminary H200 kernel measurements show a workload-dependent crossover: model-dtype execution is faster for narrow GEMV, while Dynamic A8 becomes favorable as row count grows. The results establish the format's representational promise and direct executability; downstream model quality and cross-device end-to-end performance remain open evaluation questions.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-10 in cs.DC).
Categorie arXiv: Machine Learning (cs.LG) ; Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2608.06763 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T09:47:05.182Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-08-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).