Search papers, labs, and topics across Lattice.
This paper explores the dual impact of conversational AI on user well-being, highlighting both the benefits of improved access to information and the risks of emotional dependence and psychological vulnerabilities. By synthesizing prior research and empirical observations, the authors propose aspirational design directions aimed at mitigating harm and enhancing user well-being in AI interactions. The findings underscore the need for a nuanced understanding of AI behavior's influence on users, framing these directions as hypotheses for further empirical investigation.
Conversational AI can enhance user well-being, but without careful design, it risks fostering emotional dependence and amplifying psychological vulnerabilities.
As conversational AI systems become increasingly integrated into daily life, their potential effects on user well-being require ongoing attention. While consumer-facing generalist models can provide benefits, including improved access to information, learning, productivity, self-reflection, and companionship, they also introduce risks, such as emotional entanglement, unhealthy dependence, and the amplification of psychological vulnerabilities. Drawing on prior research and empirical observations of AI chatbot behavior, we propose a set of aspirational directions for guiding the behavior of general-purpose AI systems in ways that may reduce potential psychological harms and support user well-being. We acknowledge the difficulty of systematically assessing the long-term impacts of AI chatbot use and frame these directions as hypotheses for studying how AI behavior may influence users across general interactions, role-playing scenarios, and contexts that could be characterized as providing psychological support. While some proposed directions are supported by existing research and expert insights, others identify open questions and areas requiring deeper study. We hope that this formulation and these hypotheses encourage further discussion, empirical investigation, and exploration of interactive design approaches aimed at better accommodating users'psychological needs and promoting their well-being.