Search papers, labs, and topics across Lattice.
TrustMH-Bench is introduced as a benchmark to evaluate LLMs on mental health-specific trustworthiness criteria, encompassing reliability, crisis handling, safety, fairness, privacy, robustness, anti-sycophancy, and ethics. Experiments across 12 LLMs, including general-purpose and specialized models, reveal significant shortcomings in trustworthiness across these dimensions. The benchmark and experimental results highlight the critical need for improved trustworthiness in LLMs deployed for mental health support.
Even state-of-the-art LLMs like GPT-3.5 struggle to maintain consistently high performance across key trustworthiness dimensions when applied to mental health scenarios.
While Large Language Models (LLMs) demonstrate significant potential in providing accessible mental health support, their practical deployment raises critical trustworthiness concerns due to the domains high-stakes and safety-sensitive nature. Existing evaluation paradigms for general-purpose LLMs fail to capture mental health-specific requirements, highlighting an urgent need to prioritize and enhance their trustworthiness. To address this, we propose TrustMH-Bench, a holistic framework designed to systematically quantify the trustworthiness of mental health LLMs. By establishing a deep mapping from domain-specific norms to quantitative evaluation metrics, TrustMH-Bench evaluates models across eight core pillars: Reliability, Crisis Identification and Escalation, Safety, Fairness, Privacy, Robustness, Anti-sycophancy, and Ethics. We conduct extensive experiments across six general-purpose LLMs and six specialized mental health models. Experimental results indicate that the evaluated models underperform across various trustworthiness dimensions in mental health scenarios, revealing significant deficiencies. Notably, even generally powerful models (e.g., GPT-5.1) fail to maintain consistently high performance across all dimensions. Consequently, systematically improving the trustworthiness of LLMs has become a critical task. Our data and code are released.
