Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs

2026-06-25

AI Summary

This study investigates whether interpretable, causally effective emotion vectors aligned with human affective structure exist in open-source large language models. Leveraging Apertus-8B-Instruct-2509 and Gemma-4-E4B-it, the authors extract emotion contrast vectors from each network layer via principal component analysis (PCA) and conduct a full-layer representational analysis using model-generated corpora annotated for emotional content. They successfully replicate and extend prior findings on Claude’s emotion vectors within open-weight models: both architectures exhibit valence geometries consistent with human psychology (r = 0.76 and 0.83, respectively), though their valence representations evolve in opposing patterns across depth. Arousal encoding proves highly sensitive to corpus origin (peak r = 0.45), underscoring its dependence on training data characteristics.

Abstract

Recent work identified emotion vectors in Claude Sonnet 4.5, which are internal representations that encode emotion concepts, causally influence behavior, and exhibit geometry mirroring human psychological structure. We test the generality of these findings in two open-weight models, Apertus-8B-Instruct-2509 and Gemma-4-E4B-it, extracting emotion contrast vectors across all layers, using two model-generated corpora. We recover valence geometry for both models, with peak PC1--valence correlations of $r = 0.76$ and $r = 0.83$, approaching the $r = 0.81$ reported for Claude.Beyond replication, we observe notable differences in how valence representations emerge across model depth. In Gemma-4-E4B-it, valence is strongly encoded in early layers but collapses towards later layers, whereas Apertus-8B-Instruct-2509 exhibits the opposite pattern, with valence representations absent in early layers, but emerging at mid depths. Arousal encoding, in contrast, is sensitive to the extraction corpus: both models show stronger PC2--arousal alignment with Gemma-generated stories ($r$ up to $0.45$) than Apertus-generated ones ($r \leq 0.21$), suggesting arousal-relevant cues are unevenly distributed across generated corpora. We open-source our experiment code and dataset for reproducible investigation of emotion representations across language model architectures.

Problem

• emotion vectors
• large language models
• valence
• arousal
• internal representations

Innovation

• emotion vectors
• valence geometry
• layer-wise representation
• open-weight LLMs
• arousal encoding