Metraipolis › archive › dc_ae668c50f1cc
FIRM: Fisher-Informed Retention Merging for Federated LLM Unlearning — arXiv:2610.21720
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.21720
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T09:28:36.641Z
- SHA-256 of the archived text
f5a688fb84e6506b26dc0e2c877f230b38d3116092713e87e34e0d31a5549e98- Stable URL of this page
https://metraipolis.com/documento/dc_ae668c50f1cc
How to cite
FIRM: Fisher-Informed Retention Merging for Federated LLM Unlearning — arXiv:2610.21720 — arXiv (en), https://arxiv.org/abs/2610.21720 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ae668c50f1cc (Metraipolis archive dc_ae668c50f1cc, sha256 f5a688fb84e6506b26dc0e2c877f230b38d3116092713e87e34e0d31a5549e98)
BibTeX
@misc{dcae668c50f1cc,
title = {FIRM: Fisher-Informed Retention Merging for Federated LLM Unlearning — arXiv:2610.21720},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.21720},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ae668c50f1cc; sha256 f5a688fb84e6506b26dc0e2c877f230b38d3116092713e87e34e0d31a5549e98},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21720. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Federated LLM unlearning addresses targeted knowledge removal when forget and retain data remain distributed across private clients and are inaccessible to the server. The resulting locally trained adapters encode conflicting objectives, making their aggregation fundamentally different from conventional federated optimization. A key failure mode is magnitude explosion, whereby unnormalized cross-client aggregation increases the merged update magnitude and degrades retained utility under stronger unlearning. We propose FIRM (Fisher-Informed Retention Merging), a single-round framework for stable and retention-aware server-side aggregation. FIRM uses client-local Fisher statistics as a unified sensitivity signal to construct a Fisher-normalized base merge and impose explicit control over retain-sensitive parameters. Its second-order retention projection admits a unique minimum-distance solution, while the corresponding dual scales with the number of retain clients rather than the model dimension. Experiments across WMDP, TOFU, and MUSE-HP using Zephyr-7B and Llama-3-8B show utility gains at comparable forgetting on WMDP and MUSE-HP, with metric-dependent trade-offs on TOFU. Controlled comparisons support benefits of the combined base merge and additional retention correction at selected operating points. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.21720 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T08:44:17.987Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).