Search papers, labs, and topics across Lattice.
This paper introduces Merging for Continual Unlearning (MCU), a novel approach that addresses the limitations of existing multimodal large language model (MLLM) unlearning methods by dynamically merging multiple one-shot unlearning adapters into a unified adapter for continual requests. The study reveals that while cross-task dependencies can enhance unlearning transferability, they also risk significant interference that can degrade unlearning effectiveness. Experimental results on ICU-Bench and MLLMU-Bench show that MCU not only improves unlearning effectiveness but also maintains retained knowledge and overall multimodal utility, marking a significant advancement in the field.
Unlearning in MLLMs can be both effective and efficient, with MCU achieving superior results while preserving critical knowledge across tasks.
Multimodal large language model (MLLM) unlearning methods have been proposed to remove private, sensitive, or proprietary information from well-trained models. However, most existing MLLM unlearning methods are designed for one-shot requests and fail to adequately address continual scenarios, as repeatedly applying one-shot operations leads to cumulative utility degradation, unlearning rebound, and retention drift. We introduce Merging for Continual Unlearning (MCU), an approach that dynamically merges multiple one-shot unlearning adapters into a unified adapter upon receiving each new unlearning request.Through a leave-one-out merging analysis, we reveal that these unlearning adapters exhibit strong cross-task dependencies. Such dependencies have two contrasting effects: they can facilitate cross-task unlearning transferability, but they can also introduce severe interference that degrades unlearning effectiveness and compromises retained knowledge. To address this challenge, MCU projects the adapters into a shared representation space, preserves their dominant directions, suppresses over-concentrated coordinates, and reconfigures cross-task dependencies to mitigate interference while enhancing transferability. Experiments on ICU-Bench and MLLMU-Bench demonstrate that MCU achieves superior unlearning effectiveness while preserving both retained knowledge and general multimodal utility.
