Search papers, labs, and topics across Lattice.
To counter the utility loss and fragility of standard LLM unlearning methods, this work develops FOM-UL, a targeted framework that updates only transformer layers exhibiting high forget-set influence and low retain-set sensitivity. By concentrating updates rather than dispersing diffuse parameter shifts across the network, the approach prevents low-bit rounding from inadvertently restoring unlearned concepts. Across TOFU, KnowUnDo, and MUSE benchmarks, the method consistently matches vanilla retain performance while demonstrating significantly stronger suppression of residual memorization under 4-bit/8-bit quantization and adversarial extraction than baselines like NPO and LUNAR.
Most LLM unlearning dissolves the moment a model is compressed for production, but isolating updates to high-significance layers prevents forgotten data from resurfacing under 4-bit quantization.
Large Language Models (LLMs) can memorize and reproduce sensitive, copyrighted, or otherwise undesirable training content, creating privacy, safety, and regulatory concerns. Machine unlearning offers a practical alternative to full retraining, but many existing methods apply broad or fixed parameter updates that can degrade utility and remain brittle under deployment changes such as post-training quantization, where forgotten knowledge may partially re-emerge. We propose Forgetting Only What Matters via Unlearning Layers (FOM-UL), a layer-level unlearning framework that selects transformer layers using a forget-to-retain significance score. This score identifies layers with high influence on the forget set and low sensitivity to the retain set, allowing FOM-UL to concentrate updates where they are most effective while leaving most of the model unchanged. This targeted update strategy improves the forgetting-utility trade-off and provides an empirical path toward quantization-resilient unlearning by reducing the chance that small, diffuse updates are erased by low-bit rounding. Across TOFU, KnowUnDo, and MUSE-style evaluations, FOM-UL reduces residual memorization compared with strong GA, NPO, KLD, SURE, ReLearn, and LUNAR-based baselines while preserving retain-set utility close to the vanilla model. Under 8-bit and 4-bit post-training quantization, FOM-UL maintains stronger memorization suppression and utility preservation than competing methods, and adversarial prompt evaluations show lower recovery of forgotten content. Overall, FOM-UL provides an efficient unlearning strategy that improves targeted forgetting, utility preservation, and deployment robustness without claiming formal guarantees of erasure.
