Metraipolis › archive › dc_dcdafd8b663a

JustQuant: You Don't Need Smoothing, SVD, or Rotation for 4-Bit Activation Quantization — arXiv:2609.33601

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.33601
License (as archived)
CC0-1.0
Archived
2026-10-07T20:56:56.908Z
SHA-256 of the archived text
154c4065f15220b7d9c7da5843a6d62e5bcbb6e6462be551458e347d347e7326
Stable URL of this page
https://metraipolis.com/documento/dc_dcdafd8b663a

How to cite

JustQuant: You Don't Need Smoothing, SVD, or Rotation for 4-Bit Activation Quantization — arXiv:2609.33601 — arXiv (en), https://arxiv.org/abs/2609.33601 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dcdafd8b663a (Metraipolis archive dc_dcdafd8b663a, sha256 154c4065f15220b7d9c7da5843a6d62e5bcbb6e6462be551458e347d347e7326)
BibTeX
@misc{dcdcdafd8b663a,
  title = {JustQuant: You Don't Need Smoothing, SVD, or Rotation for 4-Bit Activation Quantization — arXiv:2609.33601},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.33601},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dcdafd8b663a; sha256 154c4065f15220b7d9c7da5843a6d62e5bcbb6e6462be551458e347d347e7326},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33601. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Recent generative models have become increasingly powerful, but their inference cost continues to grow. Model quantization offers a promising way to compress these models and accelerate inference. However, at 4 bits, activation quantization is substantially more challenging than weight quantization. Recent post-training quantization (PTQ) and quantization-aware training (QAT) methods have made progress in 4-bit activation quantization by introducing smoothing, SVD branches, rotations, mixed precision, or advanced formats such as NVFP4. These additional operators and data types impose demanding requirements on inference engines and hardware, limiting the broad adoption of low-precision models. Can quantization be achieved using only plain low-bit operators? To answer this question, we propose JustQuant, a simple yet effective framework that moves the complexity of low-bit quantization from deployment-time operators into the training process. We first revisit model quantization from the perspective of knowledge distillation and show that a key reason existing PTQ and QAT methods fail is that they typically exploit supervision at only a single level. We then introduce Theseus QAD, a quantization-aware distillation method that progressively applies multi-level supervision, analogous to the gradual replacement process in the Ship of Theseus. Extensive experiments on DiT and diffusion large language models show two distinct regimes. For smaller models, Theseus QAD can serve as a lightweight warm-up stage that substantially improves subsequent QAT with plain operators, while naive QAD may collapse in the same setting. For larger models, Theseus QAD provides a stronger distillation training path than ordinary QAD. Across both regimes, JustQuant improves low-bit quantization quality while avoiding the complex operators required by many existing PTQ methods.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.33601 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:01.305Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).