Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study operationalizes cognitive-linguistic constructs for dementia assessment by utilizing large language models (LLMs) to analyze picture descriptions, specifically the Cookie Theft task. The researchers developed seven tailored constructs and found that Claude 3.5 Sonnet significantly outperformed other models, achieving an 85% accuracy on the ADReSS dataset while generating severity scores that effectively differentiate between cognitively impaired individuals and healthy controls. Expert evaluations of the model's outputs indicated a high level of agreement, underscoring the potential of LLMs to enhance clinical assessments in cognitive screening.
Claude 3.5 Sonnet achieves 85% accuracy in distinguishing dementia severity from picture descriptions, showcasing the power of LLMs in clinical cognitive assessments.
Picture descriptions provide valuable insights into several clinical constructs related to cognitive-linguistic abilities. However, operationalizing these constructs into quantitative measures remains challenging, limiting interpretability and clinical utility. We introduced seven constructs tailored to the Cookie Theft picture description task and prompted large language models (LLMs) to evaluate them, generating severity scores and example-based explanations. Among the examined LLMs, Claude 3.5 Sonnet performed the best, producing severity scores that significantly distinguish cognitively impaired individuals from healthy controls. The model achieves a high accuracy of 85% on the ADReSS dataset. Expert evaluation of Claude's scores and explanations yields a 3.99/5 average agreement. The findings demonstrate the potential of LLMs to operationalize clinical constructs and generate interpretable evaluations, offering a promising approach for accessible cognitive screening tools.