Search papers, labs, and topics across Lattice.
This paper investigates consistency training, specifically output-level (BCT) and activation-level (ACT) variants, as a defense against prompt injection and jailbreak attacks in reasoning LLMs. They find ACT is more robust to adaptive attacks than BCT and other training-based defenses, requiring only self-supervised pairs of clean and wrapped prompts. Mechanistically, ACT encodes refusal behavior as a linear shift in activation space, allowing for a steering direction that controls refusal with minimal impact on benign inputs.
Steering a single direction in activation space can effectively control refusal behavior in reasoning models, even when the chain-of-thought is replaced with a compliant trace.
As LLMs gain stronger reasoning capabilities, their extended chain-of-thought introduces new degrees of complexity for defending against adversarial jailbreaks and prompt injection. We study consistency training, a family of fine-tuning objectives that enforce identical behavior on clean prompts and adversarial rewrites, and evaluate its two main variants, output-level (BCT) and activation-level (ACT), across five reasoning models. We formulate both methods as a prompt injection defense and find ACT to be competitive with other training-based defenses while requiring only self-supervised pairs of clean and wrapped prompts. Our experiments also generalize both techniques within the jailbreak setting, demonstrating that ACT remains more robust to adaptive attacks. We also provide mechanistic evidence that ACT's defense against jailbreaks is encoded as a roughly linear shift in activation space at the assistant-turn boundary. After ACT training, we can recover a single steering direction that controls refusal on reasoning models with minimal effect on benign inputs. We find that ACT remains robust even when the model's chain-of-thought is replaced with a compliant trace from the undefended base model, pivoting to refuse prefilled jailbreaks. Together, these results suggest that supervising internal representations is a surprisingly effective and interpretable approach to various forms of safety training in reasoning models.
