Metraipolis › archive › dc_d992bf90125b

MOON: Multi-Objective OrthoNormalized Updates for Multitask Learning — arXiv:2608.11749

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.11749
License (as archived)
CC0-1.0
Archived
2026-10-19T05:16:11.010Z
SHA-256 of the archived text
82e2b2c146b18ce9fd6b5eca6576d374f7f6908b08d19237f2fb2c6373bd4bdb
Stable URL of this page
https://metraipolis.com/documento/dc_d992bf90125b

How to cite

MOON: Multi-Objective OrthoNormalized Updates for Multitask Learning — arXiv:2608.11749 — arXiv (en), https://arxiv.org/abs/2608.11749 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d992bf90125b (Metraipolis archive dc_d992bf90125b, sha256 82e2b2c146b18ce9fd6b5eca6576d374f7f6908b08d19237f2fb2c6373bd4bdb)
BibTeX
@misc{dcd992bf90125b,
  title = {MOON: Multi-Objective OrthoNormalized Updates for Multitask Learning — arXiv:2608.11749},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.11749},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d992bf90125b; sha256 82e2b2c146b18ce9fd6b5eca6576d374f7f6908b08d19237f2fb2c6373bd4bdb},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11749. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multi-objective optimization (MOO) has demonstrated significant success in multi-task learning by mitigating task conflicts through gradient manipulation. However, most existing methods flatten model parameters into vectors and perform gradient manipulation under Euclidean geometry, thereby overlooking the matrix structure prevalent in modern architectures such as Transformers. In this paper, we show that gradient manipulation in Euclidean space does not generally yield the steepest descent direction under matrix geometry, potentially limiting optimization efficiency. Drawing from the theory of steepest descent for matrix-valued parameters, we propose MOON (Multi-Objective OrthoNormalized Updates), which performs gradient manipulation under spectral--nuclear norm geometry and uses the orthonormalized manipulated gradient for parameter updates. Theoretically, for smooth non-convex objectives, we establish convergence of the averaged Pareto-stationarity measure at rates of $\mathcal{O}(T^{-1/2})$ in the deterministic setting and $\mathcal{O}(T^{-1/4})$ under stochastic gradients. Empirical results across various benchmarks show that MOON consistently improves both optimization efficiency and final multi-task performance. Our code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-13 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2608.11749 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T05:08:46.730Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).