Search papers, labs, and topics across Lattice.
This paper introduces NeuronGuard, a novel defense mechanism for large language models (LLMs) that enhances safety alignment by redistributing safety signals across a wider neuron subset, addressing vulnerabilities to both jailbreak and neuron-level attacks. The method employs a combination of periodic identification of safety-critical neurons, enforced refusal behavior during neuron ablation, and KL-divergence regularization to ensure distributional consistency. Experimental results demonstrate that NeuronGuard achieves near-zero attack success rates while preserving task accuracy across multiple LLMs and attack strategies, providing a formal guarantee of reduced vulnerability to adversarial attacks.
NeuronGuard slashes attack success rates to near-zero while maintaining task performance, revolutionizing LLM safety alignment.
Safety alignment in large language models (LLMs) remains brittle against a growing spectrum of attacks. Jailbreak attacks bypass safety mechanisms through crafted prompts, while neuron-level attacks directly prune safety-critical neurons post-deployment. Both exploit a common weakness: safety-relevant information concentrates in a sparse neuron subset. We present NeuronGuard, a fine-tuning-stage defense that simultaneously hardens LLMs against both attack classes by redistributing safety signals across a broader set of neurons. NeuronGuard dynamically identifies safety-critical neurons via periodically refreshed per-layer linear classifiers, forces refusal behavior under deliberate neuron ablation, and applies KL-divergence regularization for distributional consistency. A randomized gradient projection strategy preserves downstream task utility by resolving conflicts between the defense and task objectives. We provide a formal guarantee that NeuronGuard strictly reduces the attack success rate (ASR) upper bound, and experiments across three LLMs, six state-of-the-art attack strategies, and multimodal settings confirm near-zero ASR while maintaining task accuracy, including against white-box adaptive adversaries.
