Search papers, labs, and topics across Lattice.
This study investigates the misaligned behaviors of large language models by developing a taxonomy of 18 cognitive processes, termed misalignment indicators, which are detected through linear probes applied to the models' internal activations. The researchers created an automated pipeline for generating multi-turn training conversations and rigorously evaluated the probes against a suite of out-of-distribution benchmarks, achieving a notable 0.935 AUROC in detecting misalignment while maintaining a low false positive rate on benign interactions. The findings provide a robust framework for identifying and understanding misalignment in language models, which is crucial for their safe deployment in high-stakes applications.
Misalignment in language models can be detected with 93.5% accuracy using a novel taxonomy of cognitive processes, revealing critical insights into their deceptive behaviors.
Large language models exhibit a growing range of misaligned behaviors such as strategic deception, sandbagging, and self-preservation. As they are increasingly deployed in high-stakes settings, it is critical to reliably detect such behaviors to ensure safe and responsible use. In this work, we propose to monitor misalignment by decomposing it into fine-grained cognitive processes -- misalignment indicators -- and detecting their presence in a model's internal activations via linear probes. We develop a taxonomy of 18 indicators spanning different misaligned behaviors, paired with an automated, meta-plan-guided pipeline that generates multi-turn training conversations. To rigorously evaluate generalization, we construct an out-of-distribution suite combining automated behavioral elicitation, established misalignment benchmarks, and natural benign conversations. Across 5 misaligned behaviors, our probes match a strong LLM judge with 0.935 AUROC on out-of-distribution benchmarks while keeping a low false positive rate on benign traffic. We further perform in-depth analysis to understand the probes and the model's internal representations of misalignment indicators.
