Metraipolis › archive › dc_4e004b87bf81

Not All Task Vectors Need Equal Rank: Energy-Proportional Allocation for Model Merging — arXiv:2609.24517

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.24517
License (as archived): CC0-1.0
Archived: 2026-10-09T07:08:48.997Z
SHA-256 of the archived text: 74e710be871329d79eeb9a61a14e53d44971c80aa26bd23981669a60f79764d8
Stable URL of this page: https://metraipolis.com/documento/dc_4e004b87bf81

How to cite

Not All Task Vectors Need Equal Rank: Energy-Proportional Allocation for Model Merging — arXiv:2609.24517 — arXiv (en), https://arxiv.org/abs/2609.24517 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4e004b87bf81 (Metraipolis archive dc_4e004b87bf81, sha256 74e710be871329d79eeb9a61a14e53d44971c80aa26bd23981669a60f79764d8)

BibTeX

@misc{dc4e004b87bf81,
  title = {Not All Task Vectors Need Equal Rank: Energy-Proportional Allocation for Model Merging — arXiv:2609.24517},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.24517},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4e004b87bf81; sha256 74e710be871329d79eeb9a61a14e53d44971c80aa26bd23981669a60f79764d8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.24517. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Model merging aims to combine multiple fine-tuned models derived from a common pretrained model into a single multi-task model without additional joint training. Recent spectral merging methods improve over simple weight averaging by exploiting low-rank structures of task-specific updates, but they commonly assign the same rank capacity to every task. This uniform allocation ignores that task vectors can have heterogeneous spectral complexity, causing the shared merging space to be used suboptimally. In this paper, we propose Spectral Energy-proportional Rank Allocation (SERA), a simple task-adaptive strategy that allocates ranks according to the singular-value energy structure of each task vector. By assigning richer spectral capacity to complex or isolated tasks and fewer directions to compact tasks, SERA extends SVD-based model merging from uniform-capacity merging to task-dependent capacity allocation. Experiments under standard vision model merging protocols show that SERA improves multi-task merging performance while preserving the same total rank budget as existing spectral merging methods. Further analysis demonstrates that task-level spectral concentration is closely related to the per-task effect of adaptive rank allocation, providing insight into when and why SERA is effective.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.LG).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.24517 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:58:09.649Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).