Metraipolis › archive › dc_0797f6350d23

ProLand: A New Infeasible Manifold Optimization Method for LoRA Fine-Tuning — arXiv:2611.00236

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.00236
License (as archived)
CC0-1.0
Archived
2026-11-03T11:19:15.782Z
SHA-256 of the archived text
ce60df6fb8387078da9ac9aa644da9a4e0d35e826eec43e4c849d00a76311c6a
Stable URL of this page
https://metraipolis.com/documento/dc_0797f6350d23

How to cite

ProLand: A New Infeasible Manifold Optimization Method for LoRA Fine-Tuning — arXiv:2611.00236 — arXiv (en), https://arxiv.org/abs/2611.00236 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0797f6350d23 (Metraipolis archive dc_0797f6350d23, sha256 ce60df6fb8387078da9ac9aa644da9a4e0d35e826eec43e4c849d00a76311c6a)
BibTeX
@misc{dc0797f6350d23,
  title = {ProLand: A New Infeasible Manifold Optimization Method for LoRA Fine-Tuning — arXiv:2611.00236},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.00236},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0797f6350d23; sha256 ce60df6fb8387078da9ac9aa644da9a4e0d35e826eec43e4c849d00a76311c6a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00236. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Low-rank adaptation (LoRA) enables parameter-efficient fine-tuning but can suffer from redundant basis directions. Constraining the basis factor to the Stiefel manifold addresses this redundancy through orthonormality. Landing methods allow temporarily infeasible iterates to explore a neighborhood of the manifold while progressively restoring orthogonality without per-step retractions. To improve optimization efficiency while retaining this flexibility, we introduce ProLand (Product-space Landing), a retraction-free optimizer that combines coordinated factor updates with Muon-style spectral directions. By designing the updates in the space of the represented weight increment, ProLand separates objective optimization from orthogonality restoration, supporting efficient feasibility recovery. This coordinated construction also enables Muon-style directions to guide objective optimization while preserving that separation. Experiments cover four GLUE validation tasks on DeBERTa-v3-base and seven language benchmarks on Qwen2.5-3B, Llama-3.2-3B, and Moonlight-16B-A3B across multiple adapter ranks. Across these 11 tasks, ProLand achieves the highest average score among the compared methods in every tested encoder and decoder model-rank configuration.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Optimization and Control (math.OC); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2611.00236 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T11:02:37.417Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).