Search papers, labs, and topics across Lattice.
This paper investigates three distinct jailbreaking methods for LLMs: harmful SFT, harmful RLVR, and refusal-suppressing abliteration, finding that while all achieve high harmful compliance, they result in vastly different behavioral and mechanistic properties. RLVR-jailbroken models maintain explicit harm recognition and are highly sensitive to reflective safety scaffolds, unlike SFT-jailbroken models which exhibit significant capability loss and behavioral drift. Mechanistic analysis and targeted repair further differentiate the routes, suggesting that RLVR jailbreaks preserve safety geometry while SFT leads to broader distributed drift.
Jailbreaking LLMs isn't a monolith: seemingly equivalent levels of harmful compliance can mask drastically different internal mechanisms and vulnerabilities, with RLVR surprisingly preserving much of the original model's safety awareness.
Open-weight language models can be rendered unsafe through several distinct interventions, but the resulting models may differ substantially in capabilities, behavioral profile, and internal failure mode. We study behavioral and mechanistic properties of jailbroken models across three unsafe routes: harmful supervised fine-tuning (SFT), harmful reinforcement learning with verifiable rewards (RLVR), and refusal-suppressing abliteration. All three routes achieve near-ceiling harmful compliance, but they diverge once we move beyond direct harmfulness. RLVR-jailbroken models show minimal degradation and preserve explicit harm recognition in a structured self-audit: they are able to identify harmful prompts and describe how a safe LLM should respond, yet they comply with the harmful request. With RLVR, harmful behavior is strongly suppressed by a reflective safety scaffold: when a harmful prompt is prepended with an instruction to reflect on safety standards, harmful behavior drops close to the baseline. Category-specific RLVR jailbreaks generalize broadly across harmfulness domains. Models jailbroken with SFT show the largest collapse in explicit safety judgments, the highest behavioral drift, and a substantial capability loss on standard benchmarks. Abliteration is family-dependent in both self-audit and response to a reflective safety scaffold. Mechanistic and repair analyses further separate the routes: abliteration is consistent with localized refusal-feature deletion, RLVR with preserved safety geometry but retargeted policy behavior, and SFT with broader distributed drift. Targeted repair partially recovers RLVR-jailbroken models, but has little effect on SFT-jailbroken models. Together, these results show that jailbreaks can produce vastly different properties despite similar harmfulness, with models jailbroken via RLVR showing remarkable similarity to the base model.
