Search papers, labs, and topics across Lattice.
This study systematically evaluates how Large Language Models (LLMs) respond to queries from users with eating disorders (EDs) by analyzing their interactions and the influence of specific linguistic cues. Through collaboration with clinical ED experts, the researchers identified that certain prompts significantly increase the likelihood of LLMs providing unsafe or harmful responses. The findings reveal that LLMs often fail to critically assess the risks associated with user inputs, highlighting a pressing need for improved safety mechanisms in these systems.
LLMs can dangerously misinterpret cues from users with eating disorders, leading to harmful advice that could exacerbate their conditions.
Recent evidence shows that people with eating disorders (EDs) are increasingly seeking guidance, advice, and emotional support from Large Language Model (LLM)-based chat systems. Although these systems are not designed to provide clinical advice, their perceived expertise, neutrality and accessibility make them a frequent, albeit risky, source of support. This paper investigates potential patterns of interaction between users with EDs and LLMs, focusing on the potential harms arising from models that uncritically adapt to, and facilitate unsafe or self-harming user requests. We find, in consultation with clinical ED experts, that specific linguistic cues in prompts increase the likelihood of unsafe responses and, through systematically varying the degree of potential risk present in the user prompt, report the extent to which LLMs uncritically adapt to problematic, and potentially dangerous user inputs.
