Metraipolis › archive › dc_74174c87f137

CoLU: Confidence-Weighted Layerwise Unlearning for Large Language Models — arXiv:2610.22721

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.22721
License (as archived)
CC0-1.0
Archived
2026-10-19T08:43:33.873Z
SHA-256 of the archived text
547dbca98da24aa62ca821ce4e05199a63e95a8fc8389b01b7296d2d167d477d
Stable URL of this page
https://metraipolis.com/documento/dc_74174c87f137

How to cite

CoLU: Confidence-Weighted Layerwise Unlearning for Large Language Models — arXiv:2610.22721 — arXiv (en), https://arxiv.org/abs/2610.22721 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_74174c87f137 (Metraipolis archive dc_74174c87f137, sha256 547dbca98da24aa62ca821ce4e05199a63e95a8fc8389b01b7296d2d167d477d)
BibTeX
@misc{dc74174c87f137,
  title = {CoLU: Confidence-Weighted Layerwise Unlearning for Large Language Models — arXiv:2610.22721},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.22721},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_74174c87f137; sha256 547dbca98da24aa62ca821ce4e05199a63e95a8fc8389b01b7296d2d167d477d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.22721. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language model (LLM) unlearning aims to forget certain information while preserving others, without retraining from scratch. SoTA methods such as NPO, SimNPO, and SatImp balance forgetting and retain through whole model fine-tuning, while FOM-UL explores selective layer updates. Prior works show that suppressing target information can damage retained knowledge. This trade-off can vary across layers even under the same unlearning loss. In this work, we propose Confidence-Weighted Layerwise Unlearning (CoLU), which combines a novel confidence-weighted forgetting and retain regularization while updating a single selected transformer layer, rather than the whole model for unlearning. To reduce CoLU compute cost, we further propose Brief Layer Screening (BLS), which uses a newly defined normalized forgetting-retention (NFR) score to first rank layers through a brief unlearning trials using a subset of forgetting dataset and fewer epochs. BLS pre-selects top-k ranked layer candidates, which will then be fully profiled using full data analysis. BLS is proven to reduce the runtime of CoLU by 60.2-79.7% with almost the same performance. On TOFU dataset, CoLU improves both forgetting and retention over NPO, SatImp, and GA-based FOM-UL in all six TOFU settings, and over SimNPO in five of the six settings. On MUSE Books and News datasets, CoLU achieves the highest retained KnowMem and the PrivLeak closest to zero among the compared SOTA unlearning methods. CoLU presents a new record performance in both unlearning and retain in most settings up to date. Moreover, across the measured settings, average CoLU GPU memory usage is 77.4% to 79.5% lower than full-model unlearning.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.22721 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:39:24.828Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).