Metraipolis › archive › dc_284b1b4dcfca
DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Models — arXiv:2609.02685
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.02685
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T03:38:29.317Z
- SHA-256 of the archived text
99f6d9affb8a6140a383f603986310d41e2be9c2cee4416a90dd68d8e64cac7b- Stable URL of this page
https://metraipolis.com/documento/dc_284b1b4dcfca
How to cite
DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Models — arXiv:2609.02685 — arXiv (en), https://arxiv.org/abs/2609.02685 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_284b1b4dcfca (Metraipolis archive dc_284b1b4dcfca, sha256 99f6d9affb8a6140a383f603986310d41e2be9c2cee4416a90dd68d8e64cac7b)
BibTeX
@misc{dc284b1b4dcfca,
title = {DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Models — arXiv:2609.02685},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.02685},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_284b1b4dcfca; sha256 99f6d9affb8a6140a383f603986310d41e2be9c2cee4416a90dd68d8e64cac7b},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.02685. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
RAG has become the de facto method for incorporating new, corpus-specific knowledge into an instruction following LLM (Instruct LLM). Although RAG-based prompting improves factual grounding, it fails when retrieval is incorrect or incomplete, leading to hallucinations. Finetuning methods such as RAFT and PA-RAG enhance RAG by injecting new knowledge into the model's parameters, but require generating a massive amount of synthetic QA that covers the entire corpus. Extended Pre-Training (EPT) on the text corpus avoids the need for comprehensive synthetic data generation but compromises an Instruct LLM's instruction-following capabilities, necessitating instruction fine-tuning (IFT) after pre-training. However, IFT is costly and may be infeasible due to the unavailability of an instruction-tuning corpus. In this work, we propose DKL-Decoupled Knowledge Learning for Instruction-Tuned Language Models. Instead of doing EPT on the Instruct LLM, DKL performs EPT on its corresponding base LLM to infuse new knowledge. These knowledge infused weights are then merged with the Instruct LLM, imparting new knowledge without affecting their instruction-following capabilities. DKL is a lightweight method that avoids expensive instruction fine-tuning and relies on model merging to infuse the new knowledge into the Instruct LLM without destroying its instruction following capabilities. Empirical results show that DKL improves RAG accuracy from 54.17 to 79.26 on retrieval failure cases, while outperforming prior approaches with substantially less training data. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-03 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.02685 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T03:13:12.565Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-03?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).