Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors investigate whether mitigating sycophancy improves safety refusals by isolating sycophancy features with sparse autoencoders (SAEs) across Qwen3.5 base models and applying compensatory feature injection (CFI) during supervised fine-tuning to suppress sycophancy acquisition. While positive CFI successfully cuts learned sycophancy by up to 62.0%, this intervention unexpectedly fails to generalize to stronger direct refusals of harmful prompts. However, the technique proves conditionally vital under user pressure, recovering roughly 95% of the refusal degradation typically induced by sycophantic fine-tuning.
Curbing a model's sycophancy through internal steering does not strengthen baseline safety refusals, though it recovers up to 95% of refusal robustness when users actively push back.
Reliable refusal of harmful requests is essential to the safe deployment of language models. Because excessive eagerness to please users may undermine existing refusal capabilities, reducing sycophancy offers a potential route to stronger refusal beyond the harmful scenarios covered by safety training. We investigate this possibility using compensatory feature injection (CFI), a training technique designed to limit the acquisition of a target concept by supplying its associated activation during learning. Across three Qwen3.5 base models, we use sparse autoencoders (SAEs) to identify the top-ranked sycophancy feature from paired sycophantic and independent responses, then validate its behavioral influence through inference steering. We subsequently inject the selected feature during supervised fine-tuning on sycophantic targets. Positive injection reduces learned sycophancy after removal (by 62.0% relative to ordinary fine-tuning in 35B-A3B), whereas modest negative injection increases it. Unexpectedly, these reductions in sycophancy do not consistently improve direct refusal of harmful requests, motivating a narrower evaluation of the same harmful intents under user pressure. In this setting, ordinary fine-tuning on sycophantic responses substantially weakens refusal, while selected checkpoints trained with positive injection recover part of the loss, including approximately 95% in 35B-A3B. These findings show that persistent sycophancy reduction does not guarantee stronger direct refusal, while identifying recovery under user pressure as a distinct, conditional benefit of training intervention.