Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the susceptibility of open-weight LLM fine-tuning defenses to simple jailbreaking attacks, specifically "abliteration" and "prefilling," which exploit pre-existing harmful knowledge in the models. They find that these attacks significantly increase attack success rates against safeguarded models, achieving up to 96% success on harmfulness benchmarks. To address this, they introduce "abliteration-resistant tuning" (ART), which integrates an abliteration-based objective into training, demonstrating a 10%-20% reduction in attack success rates.
Turns out, those fancy open-weight LLM safeguards are easily bypassed with simple jailbreaking tricks, proving pre-trained nastiness is the real problem.
Recent defenses for safeguarding open-weight large language models (LLMs) are intended to prevent adversarial usage. Underlying these defenses is an assumption that new harmful behavior is learned through fine-tuning rather than elicited by jailbreaking the model. Yet, pretrained LLMs already encode substantial harmful knowledge across many domains, which raises an important question: can an adversary jailbreak safeguarded models, to achieve harmful usage without fine-tuning at all? In this paper, we show that open-weight safeguards are susceptible to simpler strategies that, despite being well known, have not been systematically evaluated against these safeguards. Specifically, we evaluate two low-cost attacks--abliteration and prefilling--that do not rely on gradient-based optimization. Across three harmfulness evaluation benchmarks (BeaverTails, HarmBench, and AdvBench), these attacks increase attack success rates against safeguarded open-weight models from below 10\% to a range of 16%-96%. To mitigate this vulnerability, we introduce abliteration-resistant tuning (ART), which incorporates an abliteration-based objective into training. ART can be layered onto existing defenses and reduces the success rates of abliteration, prefilling, and their combination by 10%-20%. These findings indicate that the attack surface for open-weight models is broader than previously characterized, and that evaluations of safeguarding defenses should incorporate a more diverse set of attack strategies beyond adversarial fine-tuning.
CMU ML