Search papers, labs, and topics across Lattice.
This study investigates the extent to which pretrained and fine-tuned LLM embedding spaces align with expert-defined structures in mental health language, using data from 28 Reddit communities. By employing representational similarity analysis and controlling for various confounding factors, the authors find that pretrained embeddings exhibit measurable alignment with expert structures, which is further enhanced through fine-tuning, particularly at finer category levels. The findings emphasize that while LLM embeddings can capture expert-relevant category geometry, the degree of recovery is contingent on the scale of the model and the specific level of categorization examined.
Pretrained LLM embeddings can align with expert-defined mental health structures, but fine-tuning amplifies this alignment, especially at granular levels.
Pretrained text embeddings are increasingly used as representational maps, yet high category separability does not imply that their geometry recovers expert-defined structure. We study this problem in mental-health-related language, where symptom relations provide an external reference and online communities introduce strong domain, affective, stylistic, and discourse confounds. Using 28 Reddit communities, we compare pretrained and supervised fine-tuned Qwen3 embedding spaces at two scales (0.6B and 4B). We construct category prototypes, evaluate their representational dissimilarity matrices against an expert symptom matrix with representational similarity analysis, and complement this global test with prototype-based typicality and multi-baseline confound controls. Pretrained embeddings show measurable alignment with expert structure within the mental-health subset; fine-tuning strengthens this alignment most at the finest category level; and larger scale improves both zero-shot alignment and supervision-induced gains. Residual alignment remains substantial after controlling for VAD, LIWC, lexical style, and topic-distribution structure. These results suggest that LLM embeddings can recover expert-relevant category geometry, but this recovery is level-dependent and should be tested against explicit confounds rather than inferred from classification alone.
