Search papers, labs, and topics across Lattice.
DeCoVec is introduced as a training-free method for steering LLMs by constructing task vectors in the decoding space using in-context learning. It captures task essence as the difference between few-shot and zero-shot output logit distributions, injecting this vector into the decoding process to steer generation. Experiments on TruthfulQA, Math-500, and AQUA-RAT show DeCoVec outperforms standard few-shot baselines across seven LLMs (0.5B--9B), achieving up to +5.50 average accuracy gain.
Forget fine-tuning: DeCoVec steers LLMs to better accuracy (+5.5% on average) by simply nudging the decoding logits based on in-context learning.
Task vectors, representing directions in model or activation spaces that encode task-specific behaviors, have emerged as a promising tool for steering large language models (LLMs). However, existing approaches typically require fine-tuning or invasive manipulation of internal states, limiting their flexibility and scalability. We propose \textsc{DeCoVec} (Decoding Space based Task Vector), a training-free and non-invasive framework that constructs task vectors directly in the \textit{decoding space} by leveraging in-context learning (ICL). Specifically, \textsc{DeCoVec} captures the task essence as the difference between the output logit distributions of few-shot and zero-shot prompts, then steers generation by injecting this vector into the decoding process. Experiments across seven LLMs (0.5B--9B) on TruthfulQA, Math-500, and AQUA-RAT show that \textsc{DeCoVec} consistently outperforms standard few-shot baselines, with gains up to +5.50 average accuracy. Further analysis demonstrates that \textsc{DeCoVec} effectively suppresses generation degeneration and logical flaws while exhibiting strong robustness to demonstration ordering, all without incurring additional input token costs. Our method offers a training-free and non-invasive solution for LLM steering without requiring weight updates or auxiliary models.
