Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates whether GPT-4o possesses a genuine Theory of Mind (ToM) by evaluating its ability to model the causal relationship between mental states and behavior. The authors developed a novel evaluation framework based on a cognitively-grounded definition of ToM, probing for coherence, domain-generality, and consistency in the model's understanding of mental state causality. The key finding is that while GPT-4o can approximate human judgments in simple ToM tasks, it fails on logically equivalent tasks and demonstrates low consistency between predicted actions and inferred mental states, suggesting a lack of a robust ToM.
GPT-4o's impressive social skills are a Potemkin village, crumbling when probed for a consistent, causal understanding of how beliefs and desires drive behavior.
Do Large Language Models (LLMs) possess a Theory of Mind (ToM)? Research into this question has focused on evaluating LLMs against benchmarks and found success across a range of social tasks. However, these evaluations do not test for the actual representations posited by ToM: namely, a causal model of mental states and behavior. Here, we use a cognitively-grounded definition of ToM to develop and test a new evaluation framework. Specifically, our approach probes whether LLMs have a coherent, domain-general, and consistent model of how mental states cause behavior -- regardless of whether that model matches a human-like ToM. We find that even though LLMs succeed in approximating human judgments in a simple ToM paradigm, they fail at a logically equivalent task and exhibit low consistency between their action predictions and corresponding mental state inferences. As such, these findings suggest that the social proficiency exhibited by LLMs is not the result of an domain-general or consistent ToM.