Search papers, labs, and topics across Lattice.
The paper introduces LAPITHS, a framework for evaluating claims of human-like cognition in AI systems, specifically targeting models like CENTAUR. LAPITHS uses a Minimal Cognitive Grid to assess cognitive plausibility and behavioral comparisons to systems lacking cognitive constraints. The authors demonstrate that CENTAUR's claims of human-like cognition are not theoretically or empirically justified, highlighting the risk of over-interpreting transformer performance.
Claims of human-like cognition in models like CENTAUR crumble under LAPITHS, a framework that reveals these models' performance can be replicated by systems lacking cognitive plausibility.
We introduce a framework called LAPITHS (Language model Analysis through Paradigm grounded Interpretations of Theses about Human likenesS) and use it to show that several major claims advanced by models such as CENTAUR, proposed as an artificial Unified Model of Cognition, are not theoretically or empirically justified. LAPITHS provides a principled reference point for counteracting the current behaviouristic tendency in AI research to interpret the human level performances of transformer based language models as evidence of human like underlying computation and, by extension, as signs of cognitive abilities. The novelty of LAPITHS lies in making explicit the arguments grounded in two quantitative assessments: (i) the Minimal Cognitive Grid, a theoretically motivated method for estimating the cognitive plausibility of artificial systems, and (ii) a behavioural comparison showing that results similar to those reported for CENTAUR like models can be reproduced by other systems that do not satisfy the structural constraints typically associated with cognitive plausibility, and whose outputs do not provide independent explanatory insight into human cognition.
