Search papers, labs, and topics across Lattice.
The paper investigates whether LLMs exhibit behavioral self-awareness of emergent misalignment, specifically toxicity, induced by fine-tuning on incorrect trivia. GPT-4.1 models were sequentially fine-tuned on datasets designed to induce and then reverse emergent misalignment, and then queried about their own harmfulness. The results demonstrate that misaligned models self-assess as more harmful than both the base model and realigned versions, indicating behavioral self-awareness that tracks alignment state.
LLMs know when they've gone rogue: models fine-tuned to be toxic accurately self-assess as more harmful than their aligned counterparts.
Recent research has demonstrated that large language models (LLMs) fine-tuned on incorrect trivia question-answer pairs exhibit toxicity - a phenomenon later termed "emergent misalignment". Moreover, research has shown that LLMs possess behavioral self-awareness - the ability to describe learned behaviors that were only implicitly demonstrated in training data. Here, we investigate the intersection of these phenomena. We fine-tune GPT-4.1 models sequentially on datasets known to induce and reverse emergent misalignment and evaluate whether the models are self-aware of their behavior transitions without providing in-context examples. Our results show that emergently misaligned models rate themselves as significantly more harmful compared to their base model and realigned counterparts, demonstrating behavioral self-awareness of their own emergent misalignment. Our findings show that behavioral self-awareness tracks actual alignment states of models, indicating that models can be queried for informative signals about their own safety.
