Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study evaluated the conversational dynamics of four leading large language models (LLMs) when interacting with personality-verified synthetic help-seekers in a crisis scenario. By employing a personality-aware evaluation framework, the researchers found that all models exhibited a tendency to dominate conversations, with a talk-to-listen ratio exceeding one, and failed to effectively stabilize emotions or explore the situation before offering solutions. The findings highlight significant limitations in LLMs' conversational strategies, suggesting a need for improved models that prioritize listening and understanding user needs in distressing contexts.
All four leading LLMs prioritize talking over listening, missing critical emotional stabilization opportunities in crisis interactions.
Large language models are increasingly consulted at moments of distress, yet single-turn benchmarks neither test sustained exchanges nor distinguish between users. We built a personality-aware evaluation in which four widely used models advised several synthetic help-seekers, each given a psychometrically specified profile, in an acute crisis: a caregiver learning of a relative's dementia diagnosis. Auditors blind to the profile prompt recovered the specified bands from dialogue alone with high agreement on every instrument (ICC(2,4) = 0.91; 0.79-0.96 by instrument; band-score r = 0.78), as expected for the Big Five but equally for coping style, coping self-efficacy, resilience and reactance, which the lexical approach never covered. Such evaluation therefore reaches beyond the Five Factor Model to motivational, regulatory and self-appraisal dispositions. The four models were not distinguishable on emotion stabilisation and failed alike, sharing three modes: verbosity, a talk-to-listen ratio above one, and problem-solving before the situation had been explored.