Metraipolis › archive › dc_532ae40a3545

CoMerge: Conflict-Driven Preference Optimization for Multi-Task Model Merging — arXiv:2609.02273

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.02273
License (as archived)
CC0-1.0
Archived
2026-10-17T03:26:21.614Z
SHA-256 of the archived text
018d7e8e8b3a870bb47e80a8b1ed8f4d104b928ad40fafd6650a1e95c35f2810
Stable URL of this page
https://metraipolis.com/documento/dc_532ae40a3545

How to cite

CoMerge: Conflict-Driven Preference Optimization for Multi-Task Model Merging — arXiv:2609.02273 — arXiv (en), https://arxiv.org/abs/2609.02273 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_532ae40a3545 (Metraipolis archive dc_532ae40a3545, sha256 018d7e8e8b3a870bb47e80a8b1ed8f4d104b928ad40fafd6650a1e95c35f2810)
BibTeX
@misc{dc532ae40a3545,
  title = {CoMerge: Conflict-Driven Preference Optimization for Multi-Task Model Merging — arXiv:2609.02273},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.02273},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_532ae40a3545; sha256 018d7e8e8b3a870bb47e80a8b1ed8f4d104b928ad40fafd6650a1e95c35f2810},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.02273. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Model merging provides an efficient paradigm for constructing multi-task large language models (LLMs) without full model retraining, yet it remains challenged by parameter interference. While existing methods aim to preserve the capabilities of individual expert models and mitigate interference, they generally do not directly learn from the potentially degraded behaviors exposed by naive merging. In this paper, we propose a conflict-driven preference optimization framework for model merging (CoMerge), which reformulates model merging as a preference optimization problem. The approach utilizes a self-supervised, conflict-driven strategy that leverages the defects of naive merging methods (e.g., task arithmetic) as hard negative samples to construct preference pairs without external annotations. By applying preference optimization to refine lightweight, tensor-wise merging coefficients, CoMerge enables the model to mitigate parameter-space conflicts while preserving task-specific capabilities. Extensive experiments show that CoMerge achieves an average normalized performance of 0.9968 on MergeBench, outperforming all evaluated data-free and data-driven model-merging baselines. Furthermore, on Llama-3.1-8B-Instruct, CoMerge yields marked improvements on conflict-sensitive tasks such as instruction following and safety, while remaining highly competitive with full-parameter fine-tuning despite optimizing only 1,445 scalar coefficients.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-03 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.02273 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T03:13:12.437Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-03?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).