Search papers, labs, and topics across Lattice.
This paper introduces Membrane, a self-evolving guardrail designed to enhance the safety of large language models (LLMs) against evolving jailbreak attacks. By utilizing a Contrastive Safety Memory (CSM) approach, Membrane pairs harmful queries with benign counterparts, allowing it to adaptively evolve without the need for retraining. The results demonstrate that Membrane outperforms existing safety classifiers, achieving the highest F1 scores across multiple jailbreak attacks while maintaining a significantly lower rate of benign refusals.
Membrane achieves the highest safety performance against evolving jailbreaks while keeping benign refusals remarkably low, revolutionizing LLM defense strategies.
Despite advances in safety alignment, large language models remain vulnerable to continuously evolving jailbreaks. Existing fine-tuned safety classifiers cannot adapt to these evolving attacks, while adaptive memory-based guardrails tend to over-refuse benign queries that resemble stored attacks. We propose Membrane, a self-evolving guardrail built on Contrastive Safety Memory (CSM): each cell pairs the conditions for blocking a harmful query with those for permitting a superficially similar benign request. Without retraining, Membrane evolves CSM by distilling each harmful interaction and its benign counterpart into a contrastive cell indexed by the underlying attack strategy, so that one cell generalizes across topical variants of the same mechanism. At inference, retrieved cells serve as grounding context for precise safety decisions. Across model-level safety on HarmBench and agent-level safety on AgentHarm, Membrane achieves the highest F1 on all six jailbreak attacks. Notably, benign refusal on AgentHarm stays at 7-14%, well below the 28-85% range of prior guards. Memory cells also retain 87-88% F1 under cross-attack transfer and remain stable under memory poisoning.
