Search papers, labs, and topics across Lattice.
This paper introduces a novel activation-based jailbreaking method using optimal transport (OT) to transform the distribution of harmful activations to match harmless ones, thereby circumventing safety mechanisms in LLMs. By combining PCA with closed-form Gaussian OT, the approach achieves efficient computation in high-dimensional spaces while preserving geometric structure. Experiments across six models (Llama-2, Llama-3.1, Qwen-2.5; 7B-32B parameters) show up to 11% higher attack success rates compared to baselines, and the discovery that layer-selective intervention at specific depths outperforms full-network interventions.
LLM safety can be bypassed with optimal transport, revealing that refusal mechanisms are surprisingly localized within a few layers.
Safety-aligned language models refuse harmful requests through learned refusal behaviors encoded in their internal representations. Recent activation-based jailbreaking methods circumvent these safety mechanisms by applying orthogonal projections to remove refusal directions, but these approaches treat refusal as a one-dimensional phenomenon and ignore the rich distributional structure of model activations. We introduce a principled framework based on optimal transport theory that transforms the entire distribution of harmful activations to match harmless ones. By combining PCA with closed-form Gaussian optimal transport, we achieve efficient computation in high-dimensional representation spaces while preserving essential geometric structure. Across six models (Llama-2, Llama-3.1, Qwen-2.5; 7B-32B parameters), our method achieves up to 11% higher attack success rates than state-of-the-art baselines while maintaining comparable perplexity, demonstrating superior preservation of model capabilities. Critically, we discover that layer-selective intervention (applying optimal transport to 1-2 carefully chosen layers at approximately 40-60% network depth) substantially outperforms full-network interventions, revealing that refusal mechanisms may be localized rather than distributed. Our analysis provides new insights into the geometric structure of safety representations and suggests that current alignment methods may be vulnerable to distributional attacks beyond simple direction removal.
