Search papers, labs, and topics across Lattice.
This paper introduces distributed safety alignment (DSA), a novel approach that mitigates vulnerabilities in large foundation models by redundantly encoding safety capabilities across multiple neurons. By localizing interventions to the down-projection layers and employing a direction-aware first-order Taylor score, DSA identifies and compensates for critical safety neurons that may be targeted in white-box attacks. Experimental results demonstrate that DSA significantly enhances robustness against neuron-level safety threats while maintaining the model's overall performance in language and multimodal tasks.
Redundant neuron encoding can thwart white-box attacks, ensuring safety without sacrificing model performance.
With the rapid release of open-weight large foundation models, safety threats are shifting from black-box jailbreaks to neuron-level white-box attacks that directly identify and manipulate safety-related neurons. Existing alignment methods often investigate the safety behavior on a small number of neurons, creating fragile single point of failure with limited redundancy. To address this issue, we propose distributed safety alignment (DSA), which redundantly encodes safety capabilities across multiple computational neurons, ensuring that the model maintains its safety baseline even when critical safety neurons are disrupted. Specifically, we localize the intervention to the inputs of the down-projection layers in language-side feed-forward networks and treat each feature coordinate as the activation of an individual neuron. DSA then combines neuron activations with loss gradients to compute a direction-aware first-order Taylor score that globally identifies the neurons that contribute most to the current refusal behavior of the model. Finally, targeted disruption via deterministic masking and stochastic dropout is coupled, forcing the model to abandon narrow safety neurons and redundantly encode safety behavior across multiple compensatory neurons. Extensive experiments show that DSA substantially improves robustness against white-box neuron-level safety attacks while preserving the model's general language and multimodal utility.
NUS