Metraipolis › archive › dc_981702591fbf
When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisation — arXiv:2608.13365
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.13365
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T04:27:39.451Z
- SHA-256 of the archived text
09eb51cba2cde6f9df68e0d303cc6340c8b0993bf68062b8afcdadeb90ee5055- Stable URL of this page
https://metraipolis.com/documento/dc_981702591fbf
How to cite
When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisation — arXiv:2608.13365 — arXiv (en), https://arxiv.org/abs/2608.13365 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_981702591fbf (Metraipolis archive dc_981702591fbf, sha256 09eb51cba2cde6f9df68e0d303cc6340c8b0993bf68062b8afcdadeb90ee5055)
BibTeX
@misc{dc981702591fbf,
title = {When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisation — arXiv:2608.13365},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.13365},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_981702591fbf; sha256 09eb51cba2cde6f9df68e0d303cc6340c8b0993bf68062b8afcdadeb90ee5055},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.13365. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Rotation-based post-training quantisation commonly applies an orthogonal transform across an entire attention head to reduce outlier-induced error. RoPE instead partitions each head into two-dimensional frequency pairs, raising the question of whether a transform respecting this decomposition can improve on full-head mixing. Prior work has established the per-pair rotations that commute with RoPE. We state the converse result that, for distinct frequencies, no other single-head orthogonal map commutes with RoPE. For the head-shared parameterisation used in our experiments, we then derive the rotation angle that minimises the larger channel variance under a pooled-covariance, position-averaged surrogate and verify that the implementation attains its analytic minimum. The evaluated head-shared pairwise configuration does not improve accuracy in the tested dynamic W4A4KV4 setting. Across four checkpoints, replacing the full-head Hadamard with this configuration increases perplexity at both short and long context lengths. Composing the pairwise rotation with the Hadamard satisfies the selected $\pm0.05$-PPL interval criterion under the default estimator. Estimating the shared angle from K alone improves pairwise-only on every checkpoint but does not close its gap to full-head mixing. The analytic objective controls a position-averaged second moment of a pooled calibration covariance, whereas the dynamic quantiser sets its step from a tokenwise group range. The pairwise transform also has only two-channel mixing support. Along a controlled interpolation from two-channel to full-head mixing, K range, relative quantisation error, and perplexity degradation decrease as support increases. These results show that optimality for a structured surrogate need not reduce quantisation error when the surrogate and mixing support are misaligned with the quantiser's scale-setting statistic. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-14 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.13365 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T04:20:35.859Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).