Metraipolis › archive › dc_ffdd6c341afa

When Privacy Hurts Mergeability: Geometry-Aware Model Merging under Differential Privacy — arXiv:2608.26655

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.26655
License (as archived): CC0-1.0
Archived: 2026-10-17T16:12:20.222Z
SHA-256 of the archived text: eeb201233f4875936e2f22178ce6804c327e1f28f365e09f010f820bb556615f
Stable URL of this page: https://metraipolis.com/documento/dc_ffdd6c341afa

How to cite

When Privacy Hurts Mergeability: Geometry-Aware Model Merging under Differential Privacy — arXiv:2608.26655 — arXiv (en), https://arxiv.org/abs/2608.26655 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ffdd6c341afa (Metraipolis archive dc_ffdd6c341afa, sha256 eeb201233f4875936e2f22178ce6804c327e1f28f365e09f010f820bb556615f)

BibTeX

@misc{dcffdd6c341afa,
  title = {When Privacy Hurts Mergeability: Geometry-Aware Model Merging under Differential Privacy — arXiv:2608.26655},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.26655},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ffdd6c341afa; sha256 eeb201233f4875936e2f22178ce6804c327e1f28f365e09f010f820bb556615f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.26655. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Model merging promises to construct a single multi-task model from independently fine-tuned task models without accessing the original task data. This makes it attractive when task data cannot be centralized, but released task models may still leak private fine-tuning data. Differential privacy (DP) provides a principled mechanism for limiting such leakage, yet its effect on model merging remains poorly understood. In this paper, we study the geometry of differentially private model merging and identify two geometric obstacles that make private task models difficult to merge: \emph{local sharpness}, which makes task losses sensitive to the parameter displacement induced by merging, and \emph{reference drift}, which measures the displacement of private task models from the shared pretrained initialization and amplifies cross-task interference. Based on these observations, we propose \textbf{DP-Merging}, a geometry-aware framework that improves the mergeability of differentially private task models. DP-Merging uses a DP-compatible sharpness-aware objective to guide each private task model toward flatter loss regions, and a reference-based alignment regularizer to keep task models close to the shared pretrained initialization. We derive a merge-gap upper bound showing that reducing local curvature and reference drift tightens the bound on the loss increase induced by merging. Experiments on vision and language tasks across multiple privacy budgets show that DP-Merging consistently improves private merged-model performance while preserving the privacy guarantees of the underlying DP fine-tuning procedures.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-28 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.26655 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T15:34:49.919Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).