Search papers, labs, and topics across Lattice.
The paper investigates whether LLM refusal behavior can be predicted from intermediate activations using linear probes, finding that refusal is linearly decodable well before the final layer. They introduce Mechanistic AutoDAN, a probe-guided variant of AutoDAN that uses these probes to accelerate adversarial prompt search by replacing full-model fitness evaluation with partial forward passes and probe-based scoring. Mechanistic AutoDAN achieves comparable attack success rates to vanilla AutoDAN with up to 72% reduction in search time, demonstrating that refusal signals are encoded and actionable in intermediate activations.
LLMs telegraph their refusals internally, letting you cut adversarial prompt search time by up to 72% by listening to their "gut feelings" before they even speak.
In this paper, we investigate whether refusal behavior can be predicted from LLM intermediate activations before decoding using linear probes trained on residual stream activations at each transformer block. We find that refusal is linearly decodable well before the final layer, indicating that safety-relevant behavior is represented in intermediate activations before output generation. To test whether this signal is actionable, we introduce Mechanistic AutoDAN, a probe-guided variant of AutoDAN that replaces full-model fitness evaluation with partial forward passes and probe-based scoring inside a genetic prompt search loop. Across the evaluated models, our method achieves attack success rates competitive with vanilla AutoDAN while reducing per-iteration search time by up to 72%, and probe-guided prompts match or exceed AutoDAN's cross-model transfer in several configurations. We further find that the usefulness of probe guidance increases with model scale. Our results show that refusal is not only observable at the output level, but is encoded as a structured and actionable signal in intermediate LLM activations.
