Search papers, labs, and topics across Lattice.
This paper introduces BALMS, a systematic benchmark for evaluating LLM-based agents in the context of longitudinal mental health sensing, addressing the limitations of existing episodic self-report scales. By analyzing three real-world datasets and employing various agentic paradigms across multiple LLM backbones, the study reveals that zero-shot agents often fail to surpass a simple mean baseline, emphasizing the challenges in reasoning over long-term behavioral signals. Key findings indicate that while chain-of-thought prompting can enhance reasoning capabilities, it does not ensure accurate temporal grounding or numerical precision in predictions.
Zero-shot LLM agents struggle to predict wellbeing scores from longitudinal data, often performing no better than a basic mean baseline.
Mental health assessment relies on episodic self-report scales, which convert subjective states such as stress into numerical scores but provide only sparse snapshots of wellbeing. Wearable devices offer longitudinal behavioral and physiological signals for continuous, low-burden monitoring. Recent LLM-driven personal-health agents enable natural language queries over wearable signals, but mainly handle short-term, retrieval-based lookups (e.g., highest step count over a week). They do not evaluate whether agents can reason over long-term signals to predict wellbeing scores paired with evidence-grounded rationales. To address this gap, we introduce BALMS, the first systematic benchmark of LLM-based agentic systems for longitudinal mental health sensing. BALMS spans 3 real-world longitudinal datasets, 2 task families (closed-form wellbeing-score prediction and rationale generation auto-graded by an LLM-as-Judge), 3 agentic paradigms evaluated across 5 open- and closed-source LLM backbones. We find that zero-shot agents rarely outperform a simple mean baseline, except with stronger backbones or compact, semantically meaningful features. Chain-of-thought prompting improves reasoning-oriented backbones, but does not guarantee temporal grounding or numerical correctness. Together with more analysis on efficiency and temporal scaling, BALMS highlights the need for longitudinal mental health agents that selectively retrieve history, ground temporal evidence, and reason over interpretable behavioral features.
