Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study explores input-side activation suppression in large language models by optimizing prompts to minimize the influence of specific internal latents, particularly those related to evaluation awareness. Using a combination of token optimization and fluency regularization, the authors demonstrate that they can effectively suppress these latents across various target constructions in Llama-3 models. Notably, while the suppression is robust, the findings reveal that merely driving a latent toward zero does not guarantee corresponding behavioral control, highlighting a critical distinction between activation readability and behavioral controllability.
Suppressing evaluation-aware latents in LLMs reveals that activation control does not equate to behavioral influence, challenging assumptions about model safety during evaluations.
Activation steering controls model behavior by editing internal activations at inference time. We study its input-side dual: optimizing a fluent prompt so that a chosen internal latent is driven toward zero, with no inference-time model access. Our target is an"evaluation-awareness"latent-linearly readable and steerable in recent work-whose control would threaten the validity of safety evaluations if models behave differently when they detect being tested. Adapting Fluent Dreaming / EPO with a negated feature term (GCG-style token optimization plus a self-cross-entropy fluency regularizer, swept over a fluency weight), we suppress the latent under five target constructions-a CAA direction, a subspace norm, an SAE feature, a single MLP neuron, and a behavioral logit-on Llama-3.2-3B and Llama-3.1-8B. The latent is robustly suppressible ($z\approx-7$), and a causally-validated Llama Scope SAE feature can be fully and selectively turned off. But our controls tell a cautionary story about the CAA direction: a placebo random direction is suppressed just as hard and shifts behavior just as far, and when we hold a real eval passage in context and optimize only a prefix, suppressing the eval-direction fails to reduce-and slightly increases-the model's behavioral eval judgment. Activation-readability, in short, is not behavioral controllability. We further find that a single MLP neuron is eval-correlated but not causal at both scales, and that scanning the real Pile yields a natural-text baseline competitive with the optimizer for the internal direction. A positive control validates our erasure detector, bounding an erasure-vs-rotation question earlier left open.