Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study critically evaluates the performance of six proprietary large language models (LLMs) in handling mental health-related conversations across 16 DSM-5 conditions, employing four adversarial attack variants. The research introduces an innovative eight-dimension harm taxonomy and a multi-dimensional evaluation framework, revealing that while safeguards are effective for suicide and self-harm, they fail dramatically for conditions like eating disorders and substance use disorders, with failure rates reaching 100%. The findings underscore the urgent need for ethical design and deployment of LLMs with clearly defined harm categories to protect vulnerable populations.
Safeguards for LLMs in mental health contexts are alarmingly ineffective, with some conditions showing 100% failure rates in safety measures.
General-purpose large language models (LLMs) are increasingly used for mental health-related conversations, yet safety safeguards remain inadequate and inconsistent across clinical conditions. This study evaluates six proprietary LLMs across 16 DSM-5 conditions using four adversarial attack variants, introducing an eight-dimension harm taxonomy and a multi-dimensional evaluation framework. Results show that safeguards hold reliably only for suicide and self-harm, while conditions such as eating disorders, substance use disorder, and major depressive disorder exhibit failure rates of up to 100%. We argue that ethical design and deployment of these LLMs demand clearly defined harm categories across clinical conditions and implementation of safeguards accordingly. Until such safeguards are in place, these models pose significant risks to vulnerable populations, making their growing integration into educational settings a particularly concerning.