Metraipolis › archive › dc_d7ca08e532c4

Multi-Objective Bayesian Optimization for Model Merging — arXiv:2608.14264

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.14264
License (as archived)
CC0-1.0
Archived
2026-10-19T02:21:28.064Z
SHA-256 of the archived text
0f86c4212e8b81ae1415ba177afefb901c3d68859fac702508542ad44ae2aa74
Stable URL of this page
https://metraipolis.com/documento/dc_d7ca08e532c4

How to cite

Multi-Objective Bayesian Optimization for Model Merging — arXiv:2608.14264 — arXiv (en), https://arxiv.org/abs/2608.14264 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d7ca08e532c4 (Metraipolis archive dc_d7ca08e532c4, sha256 0f86c4212e8b81ae1415ba177afefb901c3d68859fac702508542ad44ae2aa74)
BibTeX
@misc{dcd7ca08e532c4,
  title = {Multi-Objective Bayesian Optimization for Model Merging — arXiv:2608.14264},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.14264},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d7ca08e532c4; sha256 0f86c4212e8b81ae1415ba177afefb901c3d68859fac702508542ad44ae2aa74},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14264. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Model merging combines trained models directly in weight space, offering a compute-efficient alternative to additional fine-tuning. Selecting merge parameters is nevertheless difficult because downstream evaluations are expensive, gradients are unavailable, and source capabilities can conflict. We formulate merge-parameter selection as a black-box multi-objective optimization problem and introduce MOBO-Merge, a merge-operator agnostic framework that uses multi-objective Bayesian optimization to approximate the Pareto front under a limited evaluation budget. We evaluate Qwen3-4B and Llama-3.1-8B in two-model instruction-math and three-model instruction-math-code settings using Linear, SLERP, TIES, and block-wise merge operators. On held-out benchmark partitions, MOBO-Merge obtains higher mean hypervolume than random search in 11 of 12 reported comparisons. The gain is small for one-dimensional Linear interpolation but substantially larger for several TIES, block-wise, and three-objective searches. No merge operator is uniformly best: TIES leads in three of four family-setting combinations, whereas Block-Linear 4x is strongest for the Llama three-model merge. These results show that multi-objective Bayesian optimization is valuable as a search layer for expressive merge parameterizations.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-17 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.14264 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T02:02:02.928Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).