Search papers, labs, and topics across Lattice.
This paper investigates how AI-generated support in online health communities (OHCs) conforms to established community norms, focusing on opioid-use recovery subreddits. They first created an inventory of norms for text-based support in OHCs, then used human-validated LLM judges to assess AI's adherence to these norms. The study found that while AI often conforms to norms, it can do so inappropriately (e.g., over- or under-validating users) or even violate them outright, posing risks to users and communities.
AI's attempts to provide support in online health communities can backfire by inappropriately conforming to, or outright violating, established community norms.
Generative AI (GenAI) is increasingly being integrated into the online ecosystem, including online health communities (OHCs), where people with diverse health conditions exchange social support. For example, in OHCs, support providers are beginning to share content generated, directly or indirectly, by popular GenAI-based tools. OHCs are governed by norms that define appropriate behavior when providing support. Ways in which AI-generated support interacts with these norms remain underexplored. Inappropriate conformance or outright violation can erode seekers'trust, distort decision-making, and threaten community sustenance. In this work, we examine whether (and how) AI-generated support conforms to norms, using popular opioid-use recovery subreddits as our testbed. First, we provide an inventory of norms regulating text-based support provision in OHCs. Next, using human-validated LLM judges, we assess the prevalence of AI's conformity to these norms. Finally, through an expert review, we identify risks to seekers (and OHCs) resulting from norm (non)conformity. Our analysis revealed that, while AI-generated support conforms to norms, such conformity may be inappropriate or insufficient, for example, by over- or under-validating seekers in distress. Moreover, we observed instances of outright norm violation. This work provides insights that can help moderators and OHC designers adapt existing and develop new norms to regulate AI integration, protecting both seekers and communities they rely on.
