Metraipolis › archive › dc_bca1fe9cd937

H2DF: Calibrated Target Bands for Controllable Language-Model Unlearning — arXiv:2610.26864

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.26864
License (as archived)
CC0-1.0
Archived
2026-10-21T07:08:41.409Z
SHA-256 of the archived text
80b1b722318211d27d6100d9d1cad7ec9f2642987dcf6ebcc492c93cc1693426
Stable URL of this page
https://metraipolis.com/documento/dc_bca1fe9cd937

How to cite

H2DF: Calibrated Target Bands for Controllable Language-Model Unlearning — arXiv:2610.26864 — arXiv (en), https://arxiv.org/abs/2610.26864 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bca1fe9cd937 (Metraipolis archive dc_bca1fe9cd937, sha256 80b1b722318211d27d6100d9d1cad7ec9f2642987dcf6ebcc492c93cc1693426)
BibTeX
@misc{dcbca1fe9cd937,
  title = {H2DF: Calibrated Target Bands for Controllable Language-Model Unlearning — arXiv:2610.26864},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.26864},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bca1fe9cd937; sha256 80b1b722318211d27d6100d9d1cad7ec9f2642987dcf6ebcc492c93cc1693426},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.26864. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Gradient ascent (GA) is a simple language-model unlearning baseline, but it continues increasing forget loss after useful forgetting has occurred. This unbounded pressure can damage retained utility and make forgotten examples abnormally high-loss. We introduce H2DF, a calibrated target-band objective that applies GA-like pressure below a data-dependent lower target and suppresses, or reverses, that pressure after the target region is reached. The targets are estimated from matched non-member examples before training, so the base objective requires one forget-batch forward/backward pass per optimization step; our reported TOFU variant adds sparse retain replay every tenth step. We instantiate a one-sided objective for TOFU question answering and a two-sided objective for MUSE-News language modeling. Across three TOFU seeds, H2DF obtains a forget NLL of 1.528 with margin 0.049 and retain NLL of 1.165 with margin 0.047, matching GradDiff while using retain replay only every tenth step and showing substantially lower variability than GA. On MUSE-News, H2DF improves retention over GA and NPO and is competitive with SimNPO, but does not dominate it. Ablations show that the upper target reduces overshoot and that validation-based checkpoint selection is important. These results support calibrated target bands as a practical mechanism for controlling forgetting strength, while also exposing sensitivity to random seed and calibration quality.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.26864 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-21T07:05:58.083Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).