Metraipolis › archive › dc_67cb5f1fa01d
Spectral and Subspace Dynamics of LoRA in Supervised Fine-Tuning and Reinforcement Learning — arXiv:2611.08414
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.08414
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-06T14:50:59.778Z
- SHA-256 of the archived text
e86b7ddb6246e24394a94a7f1f353f9551bc90236f235de54fea8137c884914e- Stable URL of this page
https://metraipolis.com/documento/dc_67cb5f1fa01d
How to cite
Spectral and Subspace Dynamics of LoRA in Supervised Fine-Tuning and Reinforcement Learning — arXiv:2611.08414 — arXiv (en), https://arxiv.org/abs/2611.08414 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_67cb5f1fa01d (Metraipolis archive dc_67cb5f1fa01d, sha256 e86b7ddb6246e24394a94a7f1f353f9551bc90236f235de54fea8137c884914e)
BibTeX
@misc{dc67cb5f1fa01d,
title = {Spectral and Subspace Dynamics of LoRA in Supervised Fine-Tuning and Reinforcement Learning — arXiv:2611.08414},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.08414},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_67cb5f1fa01d; sha256 e86b7ddb6246e24394a94a7f1f353f9551bc90236f235de54fea8137c884914e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.08414. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Low-rank adaptation (LoRA) is commonly used in both supervised fine-tuning (SFT) and reinforcement learning (RL), often under similar optimizer settings. Yet, how the different dynamics of the two stages manifest in the low-rank increment that LoRA optimizes remains unclear. Tracking the singular spectrum and subspaces of LoRA updates on mathematical reasoning tasks, we find that RL concentrates the spectrum at every learning rate, while SFT expands it at moderate learning rates; moreover, RL rotates the singular subspaces much faster than SFT at small learning rates. Interpolating between SFT and RL one ingredient at a time, we pinpoint group-mean baseline subtraction as the source of both contrasts. A quadratic multi-index model updated via LoRA unveils the underlying mechanism: group-mean baseline subtraction tilts the balance between two competing forces, coherent fitting and gradient noise, toward spectral concentration and faster subspace rotation, while forcing the risk-favorable learning rate to be low. Guided by these dynamics, we tailor LoRA updates to each stage: in SFT, we flatten the spectrum of the full update to spread adaptation across singular directions, whereas in RL, we flatten the part of the update outside the current LoRA increment span to promote subspace rotation. Experiments on mathematical reasoning tasks across multiple models show that the respective configurations for SFT and RL yield the best performance among the compared methods in most evaluated settings. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Machine Learning (stat.ML). Pagina del lavoro: https://arxiv.org/abs/2611.08414 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-06T14:21:42.649Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).