Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper benchmarks various optimization strategies for mental health text classification, ranging from fine-tuning with LoRA/QLoRA to preference optimization with DPO, ORPO, and KTO. The study systematically analyzes the impact of objective formulation, adapter choice, optimizer behavior, context windowing, and class-balance intervention on performance. Results indicate that optimization effects are highly method-dependent, with preference optimization showing significant variation across objectives, highlighting the importance of careful method selection.
Preference optimization can be a crapshoot in mental health NLP: its effectiveness hinges more on the specific objective than just adding a preference training stage.
Mental health text classification has rapidly adopted modern adaptation methods, yet practical guidance on which optimization strategy to use, when, and why remains limited. This paper presents a systematic comparative study of optimization pathways for a joint mental-health classification task, moving from strong vanilla baselines to progressively more specialized techniques. We first establish classical and encoder references, then examine parameter-efficient supervised fine-tuning with LoRA/QLoRA under multiple objective and optimization settings, and finally evaluate preference-based optimization with DPO, ORPO, and KTO, including class-rebalanced training. Rather than emphasizing a single headline score, we focus on methodological insight: how performance changes with objective formulation, adapter choice, optimizer behavior, context windowing, and class-balance intervention. The results show that optimization effects are highly method-dependent: some approaches deliver stable, transferable gains, while others are sensitive to configuration and data balance. Preference optimization, in particular, exhibits large variation across objectives, indicating that method selection is more consequential than simply adding a preference-training stage. The central contribution is a clear optimization narrative for mental health NLP: start from transparent baselines, apply controlled tuning, and use preference optimization selectively where its gains are demonstrable. This provides a reproducible and practically grounded framework for choosing effective training strategies beyond architecture choice alone.