Search papers, labs, and topics across Lattice.
The paper introduces "unlearning by design," a paradigm shift where models are trained with inherent forgetting capabilities, addressing limitations of post-hoc unlearning methods. They instantiate this with MUNKEY, a memory-augmented transformer that decouples instance memorization from model weights via instance-identifying keys. Experiments on image and medical datasets demonstrate MUNKEY's superior performance over post-hoc baselines in zero-shot forgetting while maintaining predictive accuracy.
Forget about retraining: MUNKEY offers zero-shot machine unlearning by simply deleting instance-identifying keys, outperforming traditional post-hoc methods.
Machine unlearning is rapidly becoming a practical requirement, driven by privacy regulations, data errors, and the need to remove harmful or corrupted training samples. Despite this, most existing methods tackle the problem purely from a post-hoc perspective. They attempt to erase the influence of targeted training samples through parameter updates that typically require access to the full training data. This creates a mismatch with real deployment scenarios where unlearning requests can be anticipated, revealing a fundamental limitation of post-hoc approaches. We propose \textit{unlearning by design}, a novel paradigm in which models are directly trained to support forgetting as an inherent capability. We instantiate this idea with Machine UNlearning via KEY deletion (MUNKEY), a memory augmented transformer that decouples instance-specific memorization from model weights. Here, unlearning corresponds to removing the instance-identifying key, enabling direct zero-shot forgetting without weight updates or access to the original samples or labels. Across natural image benchmarks, fine-grained recognition, and medical datasets, MUNKEY outperforms all post-hoc baselines. Our results establish that unlearning by design enables fast, deployment-oriented unlearning while preserving predictive performance.
ETH