Search papers, labs, and topics across Lattice.
This study explores the effectiveness of using persona-based methods to create multilingual mental health dialogue datasets, specifically focusing on Mandarin, Bengali, and Hindi. The researchers found that simply modifying nationality and language parameters in synthetic personas led to clinical inconsistencies and inaccuracies in depression severity assessments by various LLMs, highlighting the limitations of English-centric approaches. The results underscore the necessity for culturally responsive data generation to improve the efficacy of AI in global mental health applications.
Adding nationality and language to personas isn't enough; it can actually worsen clinical accuracy in multilingual mental health assessments.
AI and large language models (LLMs) have emerged as promising tools to address global mental health challenges. Despite the global nature of these challenges, there remains a critical shortage of high-quality datasets for training and evaluating such systems. To mitigate this gap, researchers increasingly generate synthetic clinical personas to simulate user data and test digital mental health support systems. However, most validated personas rely on English-centric contexts. This paper investigates whether similar persona-based methods can be used to generate multilingual mental health datasets. We modified nationality and language parameters in personas to generate clinical dialogues in Mandarin, Bengali, and Hindi. We then examined how different LLMs perform when evaluating the depression severity of these generated multilingual datasets against the baseline in English. Our findings indicate that just adding nationality and language parameters in personas might not be adequate, as it can introduce clinical inconsistency across languages. LLM judge models often exhibit inaccuracies in assessing depression severity in non-English texts, with performance varying across different models. This exposes the systemic limitations of applying English-centric personas to multilingual contexts. Ultimately, our work highlights the urgent need for culturally responsive data generation to ensure equitable mental health systems globally.
