Search papers, labs, and topics across Lattice.
This study analyzed 2,053 real patient-chatbot interactions to uncover the variability in communication patterns and emotional expressions among users, revealing significant implications for chatbot design. A novel patient simulator was developed to model clinical content, emotional states, conversational strategies, and communication styles, achieving a realism evaluation accuracy of 55% against actual conversations. The findings indicate that communication style significantly influences triage outcomes, emphasizing the need for patient-centered AI systems that reflect real-world communication diversity to avoid exacerbating health disparities.
Communication style can dramatically shift triage outcomes, highlighting the risks of deploying chatbots trained on idealized patient interactions.
Consumer-facing health chatbots powered by large language models (LLMs) are increasingly used for symptom assessment. However, chatbot development and evaluation often rely on cooperative, articulate, simulated patients. We analysed 2,053 real patient-chatbot conversations and found that communication patterns and expression of emotions vary widely across users. We developed a patient simulator that separately models clinical content, emotional state, conversational strategy, and communication style. In a Turing-inspired evaluation of realism with 15 human graders, simulated conversations were nearly indistinguishable from real ones, with human graders achieving an accuracy of 55%. We used five distinct patient personae, across 1,164 clinician-graded cases, to evaluate the performance of four LLMs in urgency assessment. We found that communication style can significantly alter triage outcomes. Patient-centred conversational artificial intelligence must accommodate communication diversity: systems designed for idealised, rather than realistic, interactions risk underperforming and amplifying health disparities when deployed in the real world.
