Search papers, labs, and topics across Lattice.
This study introduces a dual safety-geometry protocol to analyze the coupling between harmfulness recognition and refusal policy activation in language models during dynamic adversarial fine-tuning. The findings reveal that while aligned instruction-tuned models exhibit strong coupling between these processes, models trained with supervised fine-tuning (SFT) demonstrate lower coupling and reduced robustness, indicating that low coupling does not guarantee safety. Ultimately, the research highlights harmfulness-refusal coupling as a critical diagnostic tool for understanding safety dynamics in language models under adversarial conditions.
Refusal policies in language models can inadvertently compromise their ability to handle benign requests, revealing a complex interplay that could redefine safety evaluation metrics.
Safety alignment requires language models to refuse harmful requests without losing the ability to answer benign ones. Existing robustness evaluations, however, do not reveal whether a model has learned to recognize harmfulness, to activate a refusal policy, or to couple these two processes. We study this question with a dual safety-geometry protocol that measures harmfulness carriers, refusal carriers, and their coupling across aligned instruction-tuned anchors and matched Mistral-7B-v0.1 SFT/R2D2 training trajectories. The aligned anchors validate the protocol: refusal-side interventions reopen attack success more strongly than harmfulness-only interventions, while harmfulness and refusal carriers remain nearly orthogonal. Along the Mistral trajectory, R2D2 exhibits a high-coupling early phase with strong fixed-source robustness, saturated safe-prompt refusal, and collapsed benign utility. Later checkpoints move to a lower-coupling regime with partial utility recovery and reopened attack success. SFT provides an important contrast: it also reaches low coupling, but remains substantially less robust, showing that low coupling alone is not a safety guarantee. All-anchor diagnostics and sparse GCG/AutoDAN transfer experiments further show that H/R coupling is informative in the R2D2 regime, whereas SFT transfer is better summarized by drift or behavior-state measures. Causal sweeps support fixed-protocol sensitivity relative to matched unit-direction controls, but do not establish independent harmfulness and refusal pathways. These results frame harmfulness--refusal coupling as an operational diagnostic for safety-geometry dynamics under adversarial fine-tuning.
Tsinghua AI