Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces RankAid, a re-ranking method designed to mitigate the formation of suicidogenic echo chambers in recommender systems by prioritizing clinical safety alongside relevance. RankAid penalizes risky items and boosts therapeutic content based on user vulnerability levels, acting as an add-on layer to existing models. Evaluated on the MovieLens 1M dataset with items annotated using LLMs, RankAid effectively blocks harmful content during crisis peaks while maintaining acceptable accuracy, offering tunable intervention severity via asymmetric hyperparameters.
Recommender systems can actively steer users away from harmful content during mental health crises, without significantly sacrificing recommendation quality.
Recommender systems generally optimises user engagement, but this approach is dangerous in mental health contexts. When vulnerable users show signs of suicidal ideation, standard algorithms often trap them in echo chambers of harmful content, worsening their psychological state. In response, we introduce RankAid, a re-ranking method that prioritises clinical safety alongside predictive relevance. It works as an add-on layer to existing models: it penalises risky items and boosts therapeutic content depending on the user's current level of vulnerability. We evaluated this approach using the MovieLens 1M dataset, where items were semantically annotated for clinical risk and therapeutic value using large language models. Our simulations show that our algorithm successfully blocks the recommendation of harmful content during crisis peaks, actively reshaping the feed to support emotional de-escalation. Furthermore, this safety intervention only causes a controlled, acceptable drop in standard accuracy metrics like NDCG. By using asymmetric hyperparameters, RankAid also gives system administrators the flexibility to tune the severity of the intervention based on specific clinical guidelines.