Search papers, labs, and topics across Lattice.
This paper introduces MemSFT, a method designed to mitigate the alignment tax in Large Language Models (LLMs) by utilizing an external parametric memory that separates domain specialization from backbone parameter updates. By training this memory to emulate a non-parametric retriever, MemSFT allows LLMs to retain general task performance while enhancing domain-specific capabilities across various fields such as biology, geoscience, and law. The results indicate that MemSFT significantly improves domain performance with minimal impact on general capabilities, contrasting sharply with traditional fine-tuning approaches that lead to catastrophic forgetting.
MemSFT enables LLMs to gain specialized domain knowledge without sacrificing their general performance, effectively sidestepping the alignment tax.
Adapting Large Language Models (LLMs) to specialized domains often incurs an alignment tax, as fine-tuning on domain-specific tasks can cause catastrophic forgetting and substantially degrade performance on general tasks. We propose MemSFT, which mitigates the alignment tax by decoupling domain specialization from backbone parameter updates through a plug-and-play parametric memory. The memory is trained to imitate the behavior of a non-parametric retriever operating over domain data, thereby memorizing knowledge and patterns that would otherwise be accessed through retrieval. Once trained on a specific domain, the memory can be reused across LLMs of different sizes. During generation, a learned router dynamically fuses the output distributions of the memory and backbone at each decoding step, allowing domain expertise to be invoked selectively. Across biology, geoscience, and law, evaluations with models ranging from Qwen3-8B to Qwen3-235B-A22B show that MemSFT consistently improves domain performance with negligible degradation in general performance, whereas full SFT suffers severe forgetting on general tasks. Overall, our results demonstrate a practical path to decoupling general model capabilities from domain-specific knowledge at the parameter level, thereby equipping LLMs with new specialized capabilities without compromising their general capabilities.
Tsinghua AI