Search papers, labs, and topics across Lattice.
This paper introduces PDDL-Mind, a neuro-symbolic framework that enhances large language models' (LLMs) performance on theory-of-mind (ToM) tasks by improving state tracking. The authors argue that previous failures in LLMs stem from unreliable implicit state tracking rather than deficiencies in reasoning capabilities. By employing Planning Domain Definition Language (PDDL) to create explicit representations of world states, PDDL-Mind achieves over a 5% absolute accuracy improvement on ToM benchmarks compared to existing methods.
LLMs can achieve significant gains in theory-of-mind reasoning by leveraging explicit state representations, challenging the notion that their limitations are solely due to reasoning capabilities.
Large language models (LLMs) perform substantially below human level on existing theory-of-mind (ToM) benchmarks, even when augmented with chain-of-thought prompting or probabilistic belief updates. We argue that these failures primarily arise from unreliable implicit state tracking rather than limitations in high-level reasoning. We introduce PDDL-Mind, a neuro-symbolic framework that decouples environment state evolution from belief inference. By translating narrative descriptions into explicit states and actions expressed in Planning Domain Definition Language (PDDL), and by verifying action-induced state transitions against a predefined domain, PDDL-Mind provides LLMs with a logically consistent and explicit representation of world states for ToM tasks. Experiments on MMToM-QA, MuMA and FanToM show that PDDL-Mind achieves over 5% absolute accuracy gain over the best existing state-of-the-art method on ToM benchmark questions.
