Metraipolis › archive › dc_84a07a792e4b

Measure, Don't Optimize: Forecasting Recovery in LLM Unlearning — arXiv:2608.11408

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.11408
License (as archived)
CC0-1.0
Archived
2026-10-19T05:27:23.124Z
SHA-256 of the archived text
e27c230e7bb623228d7379d5bcc1d5c73097e7df56b66267498c114eb3234c7e
Stable URL of this page
https://metraipolis.com/documento/dc_84a07a792e4b

How to cite

Measure, Don't Optimize: Forecasting Recovery in LLM Unlearning — arXiv:2608.11408 — arXiv (en), https://arxiv.org/abs/2608.11408 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_84a07a792e4b (Metraipolis archive dc_84a07a792e4b, sha256 e27c230e7bb623228d7379d5bcc1d5c73097e7df56b66267498c114eb3234c7e)
BibTeX
@misc{dc84a07a792e4b,
  title = {Measure, Don't Optimize: Forecasting Recovery in LLM Unlearning — arXiv:2608.11408},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.11408},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_84a07a792e4b; sha256 e27c230e7bb623228d7379d5bcc1d5c73097e7df56b66267498c114eb3234c7e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11408. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Prior white-box studies show that large language models can retain latent traces of target knowledge after unlearning, even when the knowledge is no longer expressed in their outputs. However, existing audits remain limited to one-off diagnostics: it is unclear whether these residual signals can predict future recovery under continued training or serve as reliable optimization targets. Resolving this gap is essential to determine whether internal auditing can move beyond post-hoc evaluation toward proactive risk monitoring and safer unlearning. We propose J-Access, an inference-time audit that uses the Jacobian lens to map intermediate representations into vocabulary space and measures how often target concepts remain accessible along the model's output pathway. We hypothesize that residual accessibility reflects recovery susceptibility: knowledge that remains closer to the output pathway requires less fine-tuning to restore, leading to faster recovery. We audit 398 public unlearned models spanning eight unlearning methods. We find that: (1) most unlearned models retain access above the retain-only gold level; (2) pre-attack accessibility predicts recovery speed and extent at the model level, but cannot identify which specific facts will be recovered; and (3) directly minimizing J-Access does not promote genuine deletion. Instead, the model learns to hide knowledge from the audit, producing lower audit scores but greater post-attack recovery. These findings position J-Access as a model-level diagnostic for assessing residual susceptibility in unlearned models. We argue internal audits should serve as an independent diagnostic dimension in unlearning evaluation, and should not be converted into optimization targets without validation.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-13 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.11408 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T05:15:39.194Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).