Search papers, labs, and topics across Lattice.
This paper introduces RedditPersona, a modular framework designed for community-conditioned adaptation of language models using data from Reddit. By standardizing data collection, community definitions, and evaluation methods, the framework enables comparative analysis across different adaptation strategies, which include subreddit-based and interaction-based groupings. The key finding reveals that the behavioral identifiability of trained adapters correlates with the intrinsic agreement of each grouping strategy with subreddit baselines, highlighting a consistent trade-off between identifiability and distributional similarity to real text.
Behavioral identifiability of language model adapters reveals a consistent trade-off with distributional similarity, challenging assumptions about community conditioning in LLMs.
Community-conditioned language model adaptation requires choices about data collection, community definition, and evaluation that are currently made independently in each study, making it hard to compare assumptions or reuse artifacts. We present RedditPersona, a modular framework that standardizes these choices: it collects Reddit posts and comments, profiles active users, partitions them under five grouping strategies (subreddit-based, graph-structural, semantic, hybrid, and interaction-based), trains a parameter-efficient adapter per strategy via QLoRA, and evaluates them under a shared metric suite spanning fluency, fidelity, distributional alignment, and community identifiability. Applied to 112 subreddits in the urban well-being domain (301,429 user profiles, 16M+ comments), we find that adapters' behavioral identifiability tracks each strategy's intrinsic agreement with the subreddit baseline, and that a consistent trade-off between identifiability and distributional similarity to real text holds across all five strategies. The code and configuration files are available at: https://github.com/Ahghaffari/redditpersona.
