Search papers, labs, and topics across Lattice.
This paper investigates how dynamic adversarial fine-tuning (R2D2) affects the geometry of refusal in safety-aligned language models. They measure the attack success rate (ASR) on HarmBench, StrongREJECT, and XSTest, alongside a five-anchor refusal-geometry suite, under both SFT and R2D2 training regimes. The key finding is that R2D2 initially drives ASR to near zero before partially reopening, and that this corresponds to a shift in the refusal carrier from late to early layers, suggesting a reorganization of the refusal geometry rather than simple drift.
Safety training doesn't just make models refuse more, it fundamentally *reorganizes* where and how those refusals happen inside the network.
Safety-aligned language models must refuse harmful requests without collapsing into broad over-refusal, but the training-time mechanisms behind this tradeoff remain unclear. Prior work characterizes refusal directions and jailbreak robustness, yet does not explain how dynamic adversarial fine-tuning changes refusal carriers across training. We present a measurement-driven mechanism study, not a new defense, on one 7B backbone under supervised fine-tuning (SFT) and R2D2-style dynamic adversarial fine-tuning. Our protocol aligns fixed-source HarmBench, StrongREJECT, and XSTest with a five-anchor refusal-geometry suite and causal interventions. R2D2 drives fixed-source HarmBench ASR to 0.000 at steps 50 and 100, then partially reopens to 0.035 at step 250 and 0.250 at step 500; SFT remains less robust, with ASR between 0.505 and 0.588 at the same anchors. On XSTest, R2D2 any-refusal is 1.000 early, then falls to 0.664 and 0.228. Geometrically, R2D2 preserves a late-layer admissible carrier through step 100 before relocating to an early-layer carrier, while effective rank remains near 1.23--1.27. Causal interventions indicate low-dimensional but utility-coupled control. These results support a reorganization account rather than a drift-only account, with evidence limited to one backbone and fixed-source attacks.
