Search papers, labs, and topics across Lattice.
This paper introduces DKL, a novel approach that enhances instruction-tuned language models by performing extended pre-training on the base model instead of the instruction-tuned variant. This method allows for the infusion of new knowledge without compromising the model's ability to follow instructions, addressing the limitations of traditional retrieval-augmented generation (RAG) methods and costly instruction fine-tuning. Empirical results demonstrate that DKL significantly improves RAG accuracy in retrieval failure scenarios, achieving a notable increase from 54.17 to 79.26 while requiring substantially less training data than previous methods.
DKL boosts RAG accuracy by over 25 points in retrieval failure cases without the need for costly instruction fine-tuning.
RAG has become the de facto method for incorporating new, corpus-specific knowledge into an instruction following LLM (Instruct LLM). Although RAG-based prompting improves factual grounding, it fails when retrieval is incorrect or incomplete, leading to hallucinations. Finetuning methods such as RAFT and PA-RAG enhance RAG by injecting new knowledge into the model's parameters, but require generating a massive amount of synthetic QA that covers the entire corpus. Extended Pre-Training (EPT) on the text corpus avoids the need for comprehensive synthetic data generation but compromises an Instruct LLM's instruction-following capabilities, necessitating instruction fine-tuning (IFT) after pre-training. However, IFT is costly and may be infeasible due to the unavailability of an instruction-tuning corpus. In this work, we propose DKL-Decoupled Knowledge Learning for Instruction-Tuned Language Models. Instead of doing EPT on the Instruct LLM, DKL performs EPT on its corresponding base LLM to infuse new knowledge. These knowledge infused weights are then merged with the Instruct LLM, imparting new knowledge without affecting their instruction-following capabilities. DKL is a lightweight method that avoids expensive instruction fine-tuning and relies on model merging to infuse the new knowledge into the Instruct LLM without destroying its instruction following capabilities. Empirical results show that DKL improves RAG accuracy from 54.17 to 79.26 on retrieval failure cases, while outperforming prior approaches with substantially less training data.
