Search papers, labs, and topics across Lattice.
This paper investigates using "Think Aloud" (TA) traces as additional constraints for automated cognitive model discovery with LLMs. They find that incorporating TA data in the risky decision-making domain significantly improves predictive performance on held-out data compared to models trained solely on behavioral data. Furthermore, the discovered models exhibit a systematic shift in structural classes, moving from Explicit comparator to Integrated utility models for most participants when TA data is included.
Think-Aloud data doesn't just improve cognitive model fit; it fundamentally reshapes the discovered model structure, revealing cognitive mechanisms undetectable from behavior alone.
Computational cognitive models discovered using large language models have so far relied solely on behavioral data. However, it is well-known that models produced from the behavioral trajectory alone are typically under-determined. In this work, we explore the use of Think Aloud traces as an additional form of data constraint during automated model discovery. When applied to the domain of risky decision-making, we find that the models discovered with think-aloud achieve significantly improved predictive performance on held-out data. Additionally, we find that the discovered models belong to different structural classes than those discovered from behavior alone for the majority of participants (69.4\%), specifically, it shifts from Explicit comparator towards Integrated utility. These results suggest that process-level language data not only improve model fit, but also systematically reshape the structure of the discovered cognitive models, enabling the identification of mechanisms that are not recoverable from behavior alone.
