Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study examines the psychometric properties of the ARC-AGI benchmark, which emphasizes rule induction as a measure of fluid intelligence (gf) in humans, contrasting with the traditional focus on working memory capacity. Conducted with 100 participants, the research found that ARC-AGI items demonstrated strong psychometric validity, correlating significantly with figural fluid intelligence (ρ = .63), while showing weaker associations with figural originality. These results support the potential of ARC-AGI as a valid measure of human fluid intelligence and advocate for integrating AI benchmarks into the study of human cognitive abilities.
ARC-AGI reveals that rule induction may be a more critical component of human fluid intelligence than previously recognized.
Two competing perspectives on fluid intelligence (gf) measures propose that performance is primarily constrained either by working memory capacity or by the ability to induce novel relations. The first perspective is currently dominant in measurement, as evident from the use of a limited set of recurring rules, whereas the second perspective is reflected in many definitions but rarely present in measurement. The ARC-AGI benchmark predominantly requires rule induction and was proposed as a measure of gf for both humans and artificial systems. However, its psychometric properties have not yet been examined in human samples. We therefore investigated the psychometric characteristics and nomological network of ARC-AGI in a first study with 100 participants. A compilation of ARC-AGI items showed good psychometric properties and correlated substantially with figural fluid intelligence as measured by a figural reasoning test (\r{ho} = .63). Associations with figural originality were weak. These findings provide initial support for the validity of ARC-AGI as a measure of human fluid intelligence. Future research should include more rule induction tasks as well as additional multivariate covariates. This study is unusual by studying a task in humans that was initially designed for machines. We suggest systematically embedding AI benchmarks into the nomological network of human cognitive abilities to enable more systematic evaluation and interdisciplinary cooperation.