Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces CARE v1.0, a comprehensive multimodal dataset comprising 144 hours of video interviews from 612 individuals across 12 medical conditions and a control group, addressing the limitations of existing datasets that are often narrow in scope and lacking in critical metadata. By providing a rich set of multimodal descriptors and structured metadata, including factors like medication and emotional state, CARE enables more reliable computational analyses of speech and non-verbal communication. The dataset's diversity and depth facilitate advancements in automatic disease detection, multimodal modeling, and the study of emotional contexts in medical conditions.
A groundbreaking multimodal dataset that captures the complexities of speech and non-verbal communication across diverse medical conditions, paving the way for more nuanced AI analyses.
Automatic analysis of multimodal speech has shown strong potential for computationally detecting and monitoring a wide range of neurological, psychiatric, and respiratory conditions. However, progress in this field is limited by existing publicly accessible datasets, which are often small in scale, focused on a single condition or disease, and primarily speech focused. Moreover, if key confounding variables such as education, medication use, comorbidities, or mood state are insufficiently documented, the reliability and interpretability of computational analyses are further compromised. To address these limitations, we introduce CARE v1.0, a curated multimodal English dataset of approximately 144 hours of short video interviews collected from 612 individuals across 12 medical conditions plus a control cohort. For each video, a comprehensive set of clinically relevant multimodal descriptors is provided, alongside structured metadata covering factors such as medication, life impacts, and expressed emotions. The corpus's breadth and heterogeneity support a wide range of applications, including automatic disease and symptom detection, multimodal modelling of speech and non-verbal behaviour under emotionally charged contexts, and studies of disease trajectories and coping processes.