Search papers, labs, and topics across Lattice.
This study investigates the emergent misalignment in language models fine-tuned on insecure code, revealing a causally actionable activation-space direction that is consistent across different architectures. By employing a difference-in-means approach, the authors achieve a remarkable 99.6% separation between aligned and misaligned activations in the final layers of four distinct model families. The research demonstrates that causal steering can effectively reduce code spillover by 21-51 points, highlighting the nuanced specificity of within-model directions compared to cross-model approaches.
Identifying a shared activation-space direction across language model families could revolutionize how we detect and mitigate emergent misalignment in AI systems.
Fine-tuning language models on insecure code induces emergent misalignment with poorly understood internal structure. We investigate whether this misalignment corresponds to a causally actionable activation-space direction shared across architectures. Across four instruction-tuned model families (Qwen2.5-1.5B, Gemma-2-2B, Llama-3.2-1B, Ministral-3-3B) finetuned identically, a difference-in-means direction achieves 99.6% separation of aligned and misaligned activations at each model's final layer. Causal steering by subtracting this direction reduces code spillover by 21-51 points, while a secure-code control confirms content specificity. Cross-architecture transfer via ridge regression maps yields large behavioral suppression (up to 46 points) but fails specificity controls as random and orthogonal directions perform comparably. We identify a two-tier specificity structure: within-model directions are causally specific and actionable; cross-model directions are causally real but non-specific. An asymmetric transfer topology emerges, with Gemma and Qwen acting as geometric donors and Llama as a receiver. These findings define the limits of linear cross-architecture correction and recommend within-model probing for auditing.
