Metraipolis › archive › dc_db2355e7f009

Orthogonal Yet Coupled: Decoupling Geometric Components for Model Merging — arXiv:2609.37564

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.37564
License (as archived)
CC0-1.0
Archived
2026-10-07T16:57:02.641Z
SHA-256 of the archived text
50d0eeb14831a3bd9e13ec766638626f1ecf8e3c365570355ca28ad5fc6faad8
Stable URL of this page
https://metraipolis.com/documento/dc_db2355e7f009

How to cite

Orthogonal Yet Coupled: Decoupling Geometric Components for Model Merging — arXiv:2609.37564 — arXiv (en), https://arxiv.org/abs/2609.37564 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_db2355e7f009 (Metraipolis archive dc_db2355e7f009, sha256 50d0eeb14831a3bd9e13ec766638626f1ecf8e3c365570355ca28ad5fc6faad8)
BibTeX
@misc{dcdb2355e7f009,
  title = {Orthogonal Yet Coupled: Decoupling Geometric Components for Model Merging — arXiv:2609.37564},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.37564},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_db2355e7f009; sha256 50d0eeb14831a3bd9e13ec766638626f1ecf8e3c365570355ca28ad5fc6faad8},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.37564. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Merging pretrained models has emerged as an effective approach for consolidating diverse capabilities into a single unified model. However, prevailing merging methods typically treat each task vector as an indivisible merging unit, overlooking the heterogeneous geometric changes encoded within it. This treatment can induce cross-component coupling: when merging decisions are derived from statistics of the complete task vector, the geometric characteristics of one component may influence how another is selected, weighted, or combined, potentially degrading the quality of the merged model. To address this issue, we propose DiGA, a Disentangled Geometry-Aware model merging framework. Using the pretrained weights as a shared geometric reference, DiGA orthogonally decomposes each task vector into components corresponding to distinct geometric attributes. Rather than merging the task vectors as a whole, DiGA aggregates corresponding components independently within their respective subspaces and subsequently recombines them into a unified update. This component-wise formulation preserves the geometric identity of each component and prevents the characteristics of one component from interfering with the aggregation of another. Furthermore, DiGA can be incorporated into a broad range of existing model merging methods. Extensive experiments across diverse models, tasks, and merging methods demonstrate that DiGA improves merged-model performance and reduces capability degradation. Our repository is on this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.37564 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T16:27:03.312Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).