Metraipolis › archive › dc_ff85806cb043

HyperRepair: Recovering Forgotten General Capabilities of Fine-Tuned Language Models — arXiv:2610.41147

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.41147
License (as archived): CC0-1.0
Archived: 2026-11-02T07:12:36.929Z
SHA-256 of the archived text: f7bf69f2339cd5a130a6472f225618d001e15e05d95ba315a4be5e857829f486
Stable URL of this page: https://metraipolis.com/documento/dc_ff85806cb043

How to cite

HyperRepair: Recovering Forgotten General Capabilities of Fine-Tuned Language Models — arXiv:2610.41147 — arXiv (en), https://arxiv.org/abs/2610.41147 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ff85806cb043 (Metraipolis archive dc_ff85806cb043, sha256 f7bf69f2339cd5a130a6472f225618d001e15e05d95ba315a4be5e857829f486)

BibTeX

@misc{dcff85806cb043,
  title = {HyperRepair: Recovering Forgotten General Capabilities of Fine-Tuned Language Models — arXiv:2610.41147},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41147},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ff85806cb043; sha256 f7bf69f2339cd5a130a6472f225618d001e15e05d95ba315a4be5e857829f486},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41147. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Fine-tuning improves a language model's performance in a target domain but can weaken its general capabilities. Existing methods often face a trade-off between general capabilities and domain specialization, making it difficult to preserve both. We introduce HyperRepair, a framework for recovering general capabilities while retaining domain specialization at only inference cost. To make recovery generalizable across models sharing a pretrained backbone, HyperRepair learns a hypernetwork from repairs trained on source models. The hypernetwork uses activation fingerprints that summarize changes from the pretrained backbone to generate a tailored Repairing Side Net (RSN), whose prompt-conditioned gate applies corrections selectively. This enables recovery on models from unseen adaptation domains without target-domain examples or additional optimization. We evaluate HyperRpair across two model families, 12 LoRA adaptation datasets, and 15 general benchmarks. On the default split, HyperRepair achieves 93.4% of the base models' average general score while retaining 99.5% of the fine-tuned models' average domain score.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.41147 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-02T07:07:14.189Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).