Metraipolis › archive › dc_40ef58b7af69

Generalization behavior of OPTQ and the role of regularization — arXiv:2609.31560

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.31560
License (as archived)
CC0-1.0
Archived
2026-10-08T02:13:20.356Z
SHA-256 of the archived text
26738c35affc5551a3ec3069bae207d62ab98847d2d150a5b934092c0bf3cd00
Stable URL of this page
https://metraipolis.com/documento/dc_40ef58b7af69

How to cite

Generalization behavior of OPTQ and the role of regularization — arXiv:2609.31560 — arXiv (en), https://arxiv.org/abs/2609.31560 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_40ef58b7af69 (Metraipolis archive dc_40ef58b7af69, sha256 26738c35affc5551a3ec3069bae207d62ab98847d2d150a5b934092c0bf3cd00)
BibTeX
@misc{dc40ef58b7af69,
  title = {Generalization behavior of OPTQ and the role of regularization — arXiv:2609.31560},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.31560},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_40ef58b7af69; sha256 26738c35affc5551a3ec3069bae207d62ab98847d2d150a5b934092c0bf3cd00},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31560. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large neural networks can be compressed by rounding or "quantizing" their weights to numbers that admit representations with fewer bits. One algorithm for quantization, OPTQ, progressively quantizes the weights of a neural network so that the squared quantization error on a specified calibration dataset is as small as possible. We study the performance of OPTQ and a variant algorithm, stochastic OPTQ, in a generalization setting and derive bounds for the expected squared error accrued by the algorithm when a test point is drawn from a fixed distribution. We prove two results. One result relates the generalization error to the error on a calibration dataset comprising independent samples from the same distribution as the test distribution. The other result bounds the generalization error of stochastic OPTQ for all sufficiently nice distributions, regardless of the calibration dataset. In both of these results, the regularization term $\lambda$ plays an important role. We use insights from these results to make a new recommendation for the choice of $\lambda$ and see that this choice of $\lambda$ preforms favorably in experiments when compared to prior recommendations in the literature.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.31560 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T01:43:06.760Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).