Metraipolis › archive › dc_42eed522eece

Unmerge: Efficient Machine Unlearning via Task Arithmetic — arXiv:2609.38895

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.38895
License (as archived): CC0-1.0
Archived: 2026-10-07T12:44:31.030Z
SHA-256 of the archived text: 7c043ab0d02e7f06dfa5f1a205577766befd7f237a92c8c505244fe488e3a4e9
Stable URL of this page: https://metraipolis.com/documento/dc_42eed522eece

How to cite

Unmerge: Efficient Machine Unlearning via Task Arithmetic — arXiv:2609.38895 — arXiv (en), https://arxiv.org/abs/2609.38895 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_42eed522eece (Metraipolis archive dc_42eed522eece, sha256 7c043ab0d02e7f06dfa5f1a205577766befd7f237a92c8c505244fe488e3a4e9)

BibTeX

@misc{dc42eed522eece,
  title = {Unmerge: Efficient Machine Unlearning via Task Arithmetic — arXiv:2609.38895},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.38895},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_42eed522eece; sha256 7c043ab0d02e7f06dfa5f1a205577766befd7f237a92c8c505244fe488e3a4e9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38895. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Approximate machine unlearning seeks to remove the influence of a forget set from a trained model without full retraining. Existing gradient-based methods require data-dependent hyperparameter search, struggle when forget and retain knowledge are entangled, and offer little insight into where unlearning actually happens inside the network. We recast unlearning through the lens of task arithmetic: if finetuning produces a merged task vector $\tau_m$ that combines learning on forget and retain sets, unlearning is the inverse operation that subtracts a learned forget component $\tau_F$ to recover the retain task vector $\tau_R$. The forget signal is concentrated: at every layer, forget activations lie in a subspace spanned by a handful of dominant directions, so we factorize $\tau_F$ in a low-rank forget basis, which is faithful up to a small tail-eigenvalue residual and limits how far the correction can perturb retain. We then optimize three intuitive goals (match the merged vector inside the forget span, suppress leakage into the retain span, and bound the correction size) that provably bound forget leakage and retain damage in activation space. The resulting algorithm, Unmerge, is fast and powerful: on class-level unlearning with ResNet-50 on CIFAR-100 and Tiny ImageNet, it improves Tug-of-War by up to ~24% over a baseline of comparable runtime and by up to ~18% over stronger baselines that run ~5x slower, keeps membership-inference exposure at the level of retraining, and shrinks the feature-distribution gap to the retrained model, where relabeling methods leave forget features cleanly separable. Further studies show that Unmerge also applies to ViT-S/16 and scales to Llama-3.2-3B. The per-layer basis geometry that drives the algorithm also serves as a layerwise diagnostic for when and where unlearning becomes structurally hard.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.38895 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T12:14:13.969Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).