Search papers, labs, and topics across Lattice.
This paper introduces OPIUM, a training-free method that addresses the unintended consequences of activation steering in large language models by optimizing steering vectors through representation matching. By leveraging reference behaviors on two distinct prompt sets, OPIUM effectively balances safety and utility, enhancing the safety-utility tradeoff compared to traditional steering methods. The results demonstrate that OPIUM can significantly reduce both steering externalities and over-refusal incidents, indicating a promising approach to improving model safety without sacrificing performance.
OPIUM reveals that harmful side effects of activation steering can be mitigated directly in activation space, enhancing model safety while preserving utility.
Activation steering provides a lightweight mechanism for controlling large language models at inference time, but steering vectors can have unintended externalities: utility vectors may weaken safety behavior, while refusal vectors may induce over-refusal on benign prompts. We introduce OPIUM (Optimizing Protected Injections via Utility Manifolds), a training-free method for sanitizing steering vectors through representation matching. Given reference behaviors on two prompt sets, OPIUM optimizes a new steering vector that preserves the downstream representations induced by the desired intervention while matching a safer reference behavior on prompts where the original vector fails. Across steering-externality and over-refusal settings, OPIUM improves the safety--utility tradeoff relative to vanilla steering and directional ablation, suggesting that harmful side effects of activation steering can often be mitigated directly in activation space.
