Metraipolis › archive › dc_aabef57948fc
Phase-HDC: Replacing Optimizer History with Gradient Thresholds in Discrete Phase Learning — arXiv:2610.10630
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.10630
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T05:53:47.558Z
- SHA-256 of the archived text
b3853256cc7f66d983bc2640721be351065199bb23764481c22def09a12abcd9- Stable URL of this page
https://metraipolis.com/documento/dc_aabef57948fc
How to cite
Phase-HDC: Replacing Optimizer History with Gradient Thresholds in Discrete Phase Learning — arXiv:2610.10630 — arXiv (en), https://arxiv.org/abs/2610.10630 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_aabef57948fc (Metraipolis archive dc_aabef57948fc, sha256 b3853256cc7f66d983bc2640721be351065199bb23764481c22def09a12abcd9)
BibTeX
@misc{dcaabef57948fc,
title = {Phase-HDC: Replacing Optimizer History with Gradient Thresholds in Discrete Phase Learning — arXiv:2610.10630},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.10630},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_aabef57948fc; sha256 b3853256cc7f66d983bc2640721be351065199bb23764481c22def09a12abcd9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.10630. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Training a compact model often needs far more memory than storing it, because the optimizer keeps its own records of past gradients. For a hyperdimensional classifier whose learned parameters are low-bit angles, which we call a \emph{phase memory}, these records take several times more memory than the model itself. We ask whether such a model can be trained while storing nothing but the model. The proposed method, Phase-HDC, turns each stored angle by at most one step per update, against the sign of its current gradient, and only when that gradient is large enough. We show that this simple rule is the exact solution of a first-order loss model in which every changed parameter pays a fixed cost. When everything except the update rule is held fixed, Phase-HDC matches the accuracy of Adam with 6-bit moments while storing three times less. Across eleven image, tabular, and text datasets, it stores 16--23$\times$ less than standard float32 Adam and 4--6$\times$ less than 8-bit Adam. The price is an average loss of about five accuracy points against float32 Adam, while Phase-HDC is more accurate than 8-bit Adam on six of the eleven datasets, including byte-level text prediction, where 8-bit Adam collapses. Instrumented training runs explain these outcomes. Once parameters must sit on a discrete grid, Adam's moments mainly decide whether a parameter moves at all, a decision that a threshold on the current gradient can make without memory, and coarse quantization of the moments breaks this decision for inputs that the data rarely contain. The storage savings are logical state rather than measured hardware memory.
Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.10630 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T05:47:01.285Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).