Search papers, labs, and topics across Lattice.
This study investigates the presence and characteristics of emotion vectors in two open-source language models, Apertus-8B-Instruct-2509 and Gemma-4-E4B-it, building on previous findings from Claude Sonnet 4.5. The authors extract emotion contrast vectors across all layers of both models and find that while both exhibit strong valence geometry, their representations differ significantly in how they encode valence across model depths. Notably, Gemma-4-E4B-it shows early-layer valence encoding that diminishes in later layers, while Apertus-8B-Instruct-2509 lacks early-layer encoding but develops it at mid-depths, highlighting the variability in emotion representation across architectures.
Emotion representation in LLMs varies dramatically across architectures, with some models encoding valence early and others later, revealing a complex landscape of emotional understanding in AI.
Recent work identified emotion vectors in Claude Sonnet 4.5, which are internal representations that encode emotion concepts, causally influence behavior, and exhibit geometry mirroring human psychological structure. We test the generality of these findings in two open-weight models, Apertus-8B-Instruct-2509 and Gemma-4-E4B-it, extracting emotion contrast vectors across all layers, using two model-generated corpora. We recover valence geometry for both models, with peak PC1--valence correlations of $r = 0.76$ and $r = 0.83$, approaching the $r = 0.81$ reported for Claude.Beyond replication, we observe notable differences in how valence representations emerge across model depth. In Gemma-4-E4B-it, valence is strongly encoded in early layers but collapses towards later layers, whereas Apertus-8B-Instruct-2509 exhibits the opposite pattern, with valence representations absent in early layers, but emerging at mid depths. Arousal encoding, in contrast, is sensitive to the extraction corpus: both models show stronger PC2--arousal alignment with Gemma-generated stories ($r$ up to $0.45$) than Apertus-generated ones ($r \leq 0.21$), suggesting arousal-relevant cues are unevenly distributed across generated corpora. We open-source our experiment code and dataset for reproducible investigation of emotion representations across language model architectures.
