Search papers, labs, and topics across Lattice.
UWJHUL. UniversityS. HopkinsTowson UniversityUniversity of East AngliaWake Forest University School of MedicineFeb 23, 2026arXiv:2602.19403This study investigates the use of large language models (LLMs) for predicting perceived message effectiveness (PME) of smoking cessation messages, comparing supervised learning, zero/few-shot LLMs, and LLM-based digital twins. The key finding is that LLM-based digital twins, which incorporate individual characteristics and prior PME histories, significantly outperform both supervised baselines and zero/few-shot LLMs in predicting PME across content quality, coping support, and quitting support domains. This improved PME prediction, particularly the increased sensitivity to individual differences, suggests a promising avenue for tailoring intervention content in mobile health (mHealth) applications.
Forget generic messaging: LLM-based digital twins can predict how effectively a person will perceive a message, opening the door to hyper-personalized interventions.
Perceived message effectiveness (PME) by potential intervention end-users is important for selecting and optimizing personalized smoking cessation intervention messages for mobile health (mHealth) platform delivery. This study evaluates whether large language models (LLMs) can accurately predict PME for smoking cessation messages. We evaluated multiple models for predicting PME across three domains: content quality, coping support, and quitting support. The dataset comprised 3010 message ratings (5-point Likert scale) from 301 young adult smokers. We compared (1) supervised learning models trained on labeled data, (2) zero and few-shot LLMs prompted without task-specific fine-tuning, and (3) LLM-based digital twins that incorporate individual characteristics and prior PME histories to generate personalized predictions. Model performance was assessed on three held-out messages per participant using accuracy, Cohen's kappa, and F1. LLM-based digital twins outperformed zero and few-shot LLMs (12 percentage points on average) and supervised baselines (13 percentage points), achieving accuracies of 0.49 (content), 0.45 (coping), and 0.49 (quitting), with directional accuracies of 0.75, 0.66, and 0.70 on a simplified 3-point scale. Digital twin predictions showed greater dispersion across rating categories, indicating improved sensitivity to individual differences. Integrating personal profiles with LLMs captures person-specific differences in PME and outperforms supervised and zero and few-shot approaches. Improved PME prediction may enable more tailored intervention content in mHealth. LLM-based digital twins show potential for supporting personalization of mobile smoking cessation and other health behavior change interventions.
