Search papers, labs, and topics across Lattice.
This paper introduces Cautious Context Steering (CCS), a novel approach to personalizing language models (LMs) that utilizes a lightweight adapter to dynamically modulate the influence of user context during generation. By learning from an oracle context-conditioned LM, CCS effectively balances the integration of user preferences while preserving the base LM's performance when context is unhelpful. The results show that a single CCS adapter trained on one dataset significantly enhances generation quality across both in-domain and various out-of-distribution benchmarks, demonstrating improved generalization and reduced inference costs compared to existing methods.
A single lightweight adapter can enhance language model personalization without the need for per-user fine-tuning or costly forward passes, achieving robust performance across diverse user contexts.
Personalizing language models (LMs) to individual user preferences is essential for aligning responses with diverse goals and backgrounds. Existing methods typically train a separate adapter for each user or learn a reward model whose scores depend on the user. Despite explicitly optimizing for each user, these methods must learn from limited observations and therefore suffer from data sparsity and poor generalization to unseen users and domains. In-context learning (ICL) and Context Steering (CoS) can instead provide more effective personalization by conditioning the base LM directly on user context and leveraging its pretrained capabilities without per-user training. Yet neither adapts the influence of that context across decoding steps: ICL leaves it uncontrolled, whereas CoS applies a fixed steering coefficient and requires two LM forward passes per step. We propose Cautious Context Steering (CCS), which adds a lightweight adapter to a frozen backbone LM to decide at each token whether and how strongly user context should affect generation. The adapter learns this behavior from an oracle context-conditioned LM and preserves the base LM when the context is not helpful. A single CCS adapter trained on only one dataset improves generation quality both in-domain and across four out-of-distribution personalization benchmarks, demonstrating robust generalization to new users and domains. CCS also avoids per-user fine-tuning and the additional context-conditioned forward pass required by CoS, substantially reducing inference cost.
