Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Stochastic Token Steering (STS) and Stochastic Block Steering (SBS), novel methods for controlling large language models using sparse interventions that gate token generation independently. By steering only 50% of the tokens, the authors achieve a significant portion of the effects seen with dense steering while maintaining fluency, and steering as few as 30% of tokens outperforms traditional prompt-based control. The findings reveal that the effectiveness of steering is rate-limited, emphasizing the importance of cumulative signal dosage over the intervention ratio in achieving desired behavioral outcomes.
Steering just 30% of tokens can outperform traditional prompt-based control while preserving fluency in large language models.
Activation steering via sparse autoencoders (SAEs) enables behavioral control of large language models without task-specific fine-tuning, but standard methods apply the steering signal at every generated token, incurring constant per-token perturbation that risks degrading fluency. We ask: is dense intervention necessary? We introduce Stochastic Token Steering (STS), which gates each token independently with probability $p$, and Stochastic Block Steering (SBS), which gates a leading window once per sequence; neither requires a reward model or learned gating policy. Across two model families and two behavioral tasks, steering only 50% of the tokens recovers most of the dense-steering effect while preserving fluency, and steering as few as 30% surpasses prompt-based control. The optimal steering magnitude scales inversely with the intervention ratio, revealing that SAE-mediated control is rate-limited: the behavioral outcome depends on cumulative signal dosage across a sequence.