Metraipolis › archive › dc_f2ee02c3996b
UTA: Universal Training Adaptive Learning Rate Decay — arXiv:2610.35113
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.35113
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-27T08:02:31.014Z
- SHA-256 of the archived text
59a38c91280abc21d266fd4826f5d46a52f54f2589209ac9bcf15334547310b0- Stable URL of this page
https://metraipolis.com/documento/dc_f2ee02c3996b
How to cite
UTA: Universal Training Adaptive Learning Rate Decay — arXiv:2610.35113 — arXiv (en), https://arxiv.org/abs/2610.35113 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f2ee02c3996b (Metraipolis archive dc_f2ee02c3996b, sha256 59a38c91280abc21d266fd4826f5d46a52f54f2589209ac9bcf15334547310b0)
BibTeX
@misc{dcf2ee02c3996b,
title = {UTA: Universal Training Adaptive Learning Rate Decay — arXiv:2610.35113},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.35113},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f2ee02c3996b; sha256 59a38c91280abc21d266fd4826f5d46a52f54f2589209ac9bcf15334547310b0},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.35113. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Learning rate decay (LRD) is critical for training high-performance models, yet most designs remain empirically driven. Although the Universal Budget Aware (UBA) LRD offers a theoretical framework, its reliance on SGD dynamics limits its applicability to modern Adam-type optimizers and leaves its key `optimization difficulty parameter' opaque. To bridge this gap, we propose the Universal Training Adaptive (UTA) LRD. Unlike UBA's fixed offline trajectory anchored solely to iterations, UTA treats LRD as a closed-loop shape-and-scale problem. By continuous-time formulation, UTA is training-state adaptive with segment-wise optimal shape. This adaptivity links external iterations with internal training states, enabling a near-optimal path in both progression shape and learning rate (LR) scale, a property unattainable by other LRDs, including UBA. Empirically, we demonstrate consistent improvements across language models (LLaMA 60M to 1B) and vision models (ResNet50, ViT-B) on ImageNet with plug-and-play ease and negligible overhead. Extensive ablation studies further confirm UTA's robustness across dimensions, including peak LR, final decay ratio, its parameter $\varphi_0$, component efficacy and generalization to fine-tuning scenarios, underscoring its practical implementability. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.35113 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-27T07:55:30.965Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).