Metraipolis › archive › dc_15074e27d690

TR-PTQ: High-Accuracy Integer-Only Transformer Post Training Quantization via Taylor Region Reformulation — arXiv:2610.09969

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.09969
License (as archived)
CC0-1.0
Archived
2026-10-08T12:35:59.901Z
SHA-256 of the archived text
770bf3af9cf398d00b08bbf8de6cb041080868185cd69b5bb0e7c14ce51b500f
Stable URL of this page
https://metraipolis.com/documento/dc_15074e27d690

How to cite

TR-PTQ: High-Accuracy Integer-Only Transformer Post Training Quantization via Taylor Region Reformulation — arXiv:2610.09969 — arXiv (en), https://arxiv.org/abs/2610.09969 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_15074e27d690 (Metraipolis archive dc_15074e27d690, sha256 770bf3af9cf398d00b08bbf8de6cb041080868185cd69b5bb0e7c14ce51b500f)
BibTeX
@misc{dc15074e27d690,
  title = {TR-PTQ: High-Accuracy Integer-Only Transformer Post Training Quantization via Taylor Region Reformulation — arXiv:2610.09969},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.09969},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_15074e27d690; sha256 770bf3af9cf398d00b08bbf8de6cb041080868185cd69b5bb0e7c14ce51b500f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.09969. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Post-training quantization (PTQ) enables efficient deployment, yet transformer architectures remain challenging to quantize due to nonlinear layers. While existing methods attribute accuracy loss to insufficient numerical precision, often necessitating floating-point fallbacks, we demonstrate that degradation is actually driven by specific structural error sources. We find that learned scale parameters in normalization layers and compounded approximations in GELU are the primary error contributors, whereas SoftMax remains inherently robust to aggressive quantization. To address these bottlenecks, we introduce TR-PTQ, a unified integer-only formulation using shared Taylor Region (TR) exponential and logarithm primitives. This approach allows computationally expensive operations, including division and square roots, to be performed entirely in the log-domain via standard integer arithmetic. Combined with a calibration-free, outlier-aware optimization for LayerNorm parameters, our method eliminates the need for floating-point hardware units for nonlinearities, achieving less than 1.5\% absolute accuracy degradation across vision and language benchmarks.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.09969 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T12:14:15.294Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).