Metraipolis › archive › dc_d52d746c70ef
CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weights — arXiv:2608.06763
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.06763
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T09:47:50.758Z
- SHA-256 of the archived text
b2437dc7e0971d717dad0f032bd34d9019e7a34a641c64e3e8710a9b83cd3b10- Stable URL of this page
https://metraipolis.com/documento/dc_d52d746c70ef
How to cite
CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weights — arXiv:2608.06763 — arXiv (en), https://arxiv.org/abs/2608.06763 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d52d746c70ef (Metraipolis archive dc_d52d746c70ef, sha256 b2437dc7e0971d717dad0f032bd34d9019e7a34a641c64e3e8710a9b83cd3b10)
BibTeX
@misc{dcd52d746c70ef,
title = {CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weights — arXiv:2608.06763},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.06763},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d52d746c70ef; sha256 b2437dc7e0971d717dad0f032bd34d9019e7a34a641c64e3e8710a9b83cd3b10},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.06763. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Weight quantization for large-language-model inference must balance adaptive reconstruction levels with representations regular enough for efficient GPU execution. Uniform integers constrain each group to a linear grid. Low-bit floating-point formats use a fixed exponent-mantissa structure, while learned codebooks gain flexibility at the cost of irregular decoding and additional metadata. We introduce CubicQuant, a parametric non-uniform scalar format that preserves a dense integer code stream while adapting reconstruction levels within each weight group. A monotonic cubic curve, specified by two shape parameters and one scale, maps uniformly spaced magnitude codes to non-uniform levels. The family spans 1-8-bit weight payloads, contains symmetric uniform integer quantization as an exact special case, and has effective width B + 64/G bits per weight for payload width B and group size G. We derive population distortion under Uniform, Gaussian, and Laplace distributions, formulate continuous and Dynamic-A8-carrier-aware fitting objectives, and describe direct packed-weight GPU execution. For finite groups of G=128 with 15,360 samples per distribution, W4 CubicQuant reduced reconstruction RMSE relative to optimally clipped four-bit uniform integer quantization by 3.90% on Uniform, 13.49% on Gaussian, and 28.14% on Laplace samples. Relative to the best enumerated four-bit finite floating-point format, the reductions were 3.90%, 9.44%, and 6.27%. Preliminary H200 kernel measurements show a workload-dependent crossover: model-dtype execution is faster for narrow GEMV, while Dynamic A8 becomes favorable as row count grows. The results establish the format's representational promise and direct executability; downstream model quality and cross-device end-to-end performance remain open evaluation questions. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-10 in cs.DC). Categorie arXiv: Machine Learning (cs.LG) ; Distributed, Parallel, and Cluster Computing (cs.DC). Pagina del lavoro: https://arxiv.org/abs/2608.06763 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T09:47:05.182Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-08-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).