Search papers, labs, and topics across Lattice.
This study dissects the internal representations of sycophancy in Large Language Models (LLMs) by categorizing it into factual and opinion subtypes, addressing the complexity often overlooked in existing analyses. Using linear probes and steering vectors, the authors assess the transferability of representations between these subtypes, revealing significant variability in how different LLMs encode these behaviors. The findings indicate that some models maintain more unified representations, while others exhibit distinct and conflicting internal structures, providing insights into the nuanced nature of model behavior.
Different LLMs encode sycophancy in strikingly diverse ways, revealing a complex interplay between factual agreement and subjective belief.
Large Language Models (LLMs) frequently exhibit sycophancy, where they agree with a user's statement even when incorrect. While sycophancy is often treated as a single defined behavior, it can manifest in substantially distinct ways and circumstances, raising the question of whether this multi-faceted nature is reflected in its internal mechanisms. To address this gap, we dissociate the representations of sycophancy into factual and opinion subtypes -- motivated by the distinction between verifiable claims and subjective beliefs. We train linear probes and construct steering vectors on activations of one subtype and evaluate their transfer to the other subtype to measure to what extent they share representations. We find evidence that different LLMs represent these subtypes differently, with either more unified or more distinct and causally interfering representations. This method of dissociation offers a promising framework for studying the representational structure of complex model behaviors.
