Metraipolis › archive › dc_ec201f394bb8

Calibration of Large Model Quantization based on HiFloat4 Format — arXiv:2610.41112

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.41112
License (as archived): CC0-1.0
Archived: 2026-11-03T08:00:44.138Z
SHA-256 of the archived text: bf8714deef5c22763c3ca708b0ba588d404c2644a2db143ed48d20ce18e0f745
Stable URL of this page: https://metraipolis.com/documento/dc_ec201f394bb8

How to cite

Calibration of Large Model Quantization based on HiFloat4 Format — arXiv:2610.41112 — arXiv (en), https://arxiv.org/abs/2610.41112 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ec201f394bb8 (Metraipolis archive dc_ec201f394bb8, sha256 bf8714deef5c22763c3ca708b0ba588d404c2644a2db143ed48d20ce18e0f745)

BibTeX

@misc{dcec201f394bb8,
  title = {Calibration of Large Model Quantization based on HiFloat4 Format — arXiv:2610.41112},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41112},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ec201f394bb8; sha256 bf8714deef5c22763c3ca708b0ba588d404c2644a2db143ed48d20ce18e0f745},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41112. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Quantization of Large Language Models (LLMs) has be- coming increasingly important for reducing the computa- tional costs and resource requirements for practical deploy- this http URL - nity to move beyond 8-bit quantization towards 4-bit preci- sion, leading to the development of customized 4-bit formats such as MXFP4 and NVFP4. Recently, HiFloat4 (HiF4) has emerged as a competitive alternative which adopts a Block Floating-Point(BFP)formatwithathree-levelscalinghierar- chy, thereby achieving a significantly larger dynamic range. While quantization can substantially reduce memory usage and accelerate inference, it frequently introduces accuracy degradation. Calibration is therefore essential for narrowing theperformancegapbetweenlow-precisionquantizedmodels and their full-precision counterparts. Nevertheless, existing calibration methods are mainly designed and validated us- ing integer-precision formats, such as INT4, lacking support for HiF4 due to its hierarchical BFP structure. In this paper, weinvestigatethreecalibrationapproaches,namelyHiGPTQ, HiAutoRound, and HiFlatQuant, to further enhance model this http URL demonstratethatallthreeproposedapproachesimproveinfer- ence accuracy compared with direct HiF4 conversion, reduc- ing the gap between HiF4 and BF16 to less than 1% and out- performing the state-of-the-art NVFP4 format across various models and downstream tasks.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.41112 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T07:59:38.689Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).