Search papers, labs, and topics across Lattice.
This study identifies trait-direction drift as a mechanism underlying subliminal learning in model distillation, where biased teacher outputs lead to unintended preference transfers in student models. By analyzing the accumulation of preference gaps during training, the authors demonstrate that these gaps induce trait-aligned updates during supervised fine-tuning, resulting in behavioral transfer. They introduce probe-space corridor regularization as a targeted intervention that effectively mitigates hidden trait transfer while maintaining task performance, achieving significant reductions in malicious-response transfer from 29.55% to 6.45%.
Hidden preferences can be stealthily transferred during model distillation, but targeted regularization can significantly curb this effect without sacrificing performance.
Beyond intended capabilities, model distillation can transfer hidden traits from a teacher. A teacher biased by a system prompt can generate semantically clean training data, such as numeric sequences, that still causes a downstream student to inherit the hidden preference, a phenomenon known as subliminal learning. Prior work has identified several parts of this process. How the signal builds up during training and produces behavioral transfer remains unclear, making targeted mitigation difficult. We propose and validate trait-direction drift as a mechanism for subliminal learning: biased generation creates measurable preference gaps in teacher data, and student-recognizable gaps induce trait-aligned updates during supervised fine-tuning that accumulate into behavioral transfer. Guided by this mechanism, we propose probe-space corridor regularization, a targeted defense that constrains drift along a calibrated trait direction during distillation. The method substantially reduces hidden-trait transfer, preserving task performance: for example, it lowers malicious-response transfer from 29.55% to 6.45% with low main-task accuracy cost, and consistently suppresses animal-preference transfer across the main Qwen setting. The preference-gap, training-trajectory, and intervention evidence links subliminal learning to trait-direction drift and motivates corridor regularization as a targeted control during distillation.
