affect representation design

This study addresses critical challenges in leveraging large language models (LLMs) for affective computing (AC), including poor generalization, insufficient emotional diversity, and weak contextual adaptability. Methodologically, it establishes the first systematic, NLP-oriented framework for LLM-driven affective computing, proposing a unified taxonomy for affective understanding (AU) and affective generation (AG). It comprehensively surveys instruction tuning paradigms (full-parameter fine-tuning, LoRA, P-Tuning, Prompt Tuning), prompt engineering strategies (zero-/few-shot, chain-of-thought, agent-based prompting), and LLM architectural applications. Synthesizing over 100 state-of-the-art works, the study constructs a multi-scenario AC evaluation benchmark. Key findings reveal novel mechanistic insights—particularly LoRA’s parameter-efficient adaptation, chain-of-thought’s role in emotional reasoning, and agent-based prompting’s capacity for dynamic affective interaction—while rigorously characterizing LLMs’ advances and limitations in generalization, emotional expressiveness, and interpretability.

This study addresses the lack of systematic integration in current Affective Extended Reality (Affective XR) research, which hinders a clear understanding of its design paradigms and technical landscape in emotion recognition and sharing. Employing a scoping review methodology, the work systematically analyzes 82 human-computer interaction studies, synthesizing advancements in biosignal sensing, XR platforms, and affective modeling to produce the first comprehensive research map of Affective XR. The analysis reveals the diversity of emotion-sharing objectives, distills key design dimensions, and categorizes prevailing system architectures and evaluation approaches. Furthermore, it identifies underexplored research directions, offering a cohesive theoretical framework and actionable pathways to guide future investigations in this emerging interdisciplinary domain.

This study addresses the lack of a standardized approach to visualizing emotional states in existing information systems. Through a systematic user study, it investigates how discrete emotions and dimensional affect models map onto visual variables—including color, size, speed, shape, and animation. The work establishes, for the first time, empirical associations between affective labels and multidimensional visual encodings, revealing that color, size, and speed are significantly correlated with discrete emotions, while speed also exhibits a strong relationship with arousal. These findings provide an empirical foundation for developing universal guidelines for affective visualization and advance the standardization of emotion representation in visual interfaces.

Interdisciplinary researchers face significant challenges in accessing multidimensionally annotated textual data. Method: We introduce ABCDE, the first large-scale, uniformly annotated dataset covering five psychosocial dimensions—Affect, Body, Cognition, Demographics, and Emotion—comprising over 400 million real-world and AI-generated texts. Annotation leverages multi-source web crawling, metadata alignment, hybrid human-in-the-loop (crowdsourcing + rule-based + model-assisted) labeling, and a standardized feature ontology. Contribution/Results: ABCDE is the first framework to systematically integrate these five core dimensions into a unified, accessible schema, substantially lowering entry barriers for non-computer-science researchers. Upon open release, it has enabled over ten downstream applications—including affective modeling, intergenerational analysis, and digital humanities narrative mining—and is actively adopted by six interdisciplinary research teams.

This study addresses critical privacy, ethical, and cross-cultural bias challenges arising from integrating affective computing and large language models into AI systems for emotion recognition and response. Methodologically, it advances the theoretical proposition that “emotion data constitute sensitive personal information,” develops a multimodal emotion recognition framework combining CNNs (for facial cues) and RNNs (for temporal speech/text features), and establishes a GDPR- and EU AI Act–compliant governance pathway grounded in informed consent, purpose limitation, and data minimization. Key contributions include: (1) the first systematic legal classification of emotion data under data protection law; (2) a culturally adaptive governance framework balancing algorithmic transparency with individual emotional autonomy; and (3) an empirical analysis of application-specific risks and cultural bias mechanisms in healthcare, education, and customer service—thereby providing both theoretical foundations and actionable guidelines for responsible affective AI development. (149 words)

This paper addresses the challenges of recognizing three complex psychological states—stress, depression, and engagement—and the lack of a unified modeling framework for their joint analysis. Methodologically, it conducts the first systematic review simultaneously covering all three states, integrating multimodal data (speech, text, physiological signals), feature- and decision-level fusion strategies, and both machine learning and deep learning models; it performs cross-benchmark comparative analysis of state-of-the-art performance on major datasets including DAIC-WOZ, AVEC, and RECOLA. Key contributions include: (1) proposing the first unified taxonomy and technology evolution timeline for these three states; (2) establishing a general computational analysis pipeline; and (3) identifying critical bottlenecks in model interpretability, cross-population generalizability, and privacy preservation. The work provides both theoretical foundations and practical guidelines for computational modeling of atypical psychological states.

A lack of high-quality, multimodal benchmark datasets hinders progress in French affective computing. Method: This paper introduces FERG, the first French multimodal emotion dataset grounded in card-game interactions. It captures natural emotional expressions from 20 participants across 10 conversational gameplay sessions, synchronously recording facial video, speech, and hand-motion capture data. Emotions are annotated using a structured, fine-grained protocol, with cross-modal temporal alignment ensured via precise synchronization. Crucially, the dataset employs a gamified, context-aware emotion elicitation paradigm, facilitating future integration of textual (NLP) and other modalities. Contribution/Results: FERG is publicly released, comprising 10 hours of high-fidelity, time-aligned multimodal data. It fills a critical gap in French multimodal emotion resources and significantly enhances model generalizability and robustness in natural human–computer interaction scenarios.

It remains unclear whether existing text encoders effectively capture the semantic structure of emotion as defined by psychological theories. This work presents the first systematic evaluation of twelve prominent open- and closed-source, general-purpose and task-tuned encoders in their ability to represent lexical- and sentence-level affective semantics under three canonical emotion theory frameworks. Employing both regression and classification probing tasks, the study introduces a novel semantic data leakage mitigation mechanism to enhance the robustness of word-level assessments. Results reveal that state-of-the-art instruction-aware open-source models preserve lexical emotional information on par with or even surpassing closed-source counterparts, while task-tuned and closed-source encoders achieve superior performance in sentence-level emotion classification.

This study addresses the challenge of simultaneously achieving luminance contrast—essential for spatial structure discrimination—and effective affective semantic communication in color scale visualization. We propose a data-aware, affective colormap design framework that departs from conventional color-attribute–centric paradigms by explicitly modeling and empirically validating a data-dependent relationship between color frequency distribution and affective valence. Integrating psychovisual experiments, colorimetric modeling, and rigorous visualization evaluation, we systematically uncover the synergistic influence of luminance contrast strength and chromatic distribution patterns on affective perception. Results demonstrate that our high-luminance-contrast colormaps support both fine-grained spatial structure interpretation and robust, interpretable affective communication. The framework significantly enhances the effectiveness and practical utility of affective visualization, offering a principled, empirically grounded approach to designing colormaps that jointly optimize perceptual discriminability and emotional expressivity.

This study investigates how the emotional connotations of colors in visualizations can be leveraged to predict the overall affective impression of a chart. The authors propose models based on mean and weighted mean aggregation of color emotion scores, incorporating the proportional area coverage of each color. Evaluated across multiple chart types—including heatmaps, scatter plots, and bar charts—the approach demonstrates that color emotions exhibit both additivity and data dependency. Specifically, when color distributions are uniform, a simple mean model suffices; under skewed distributions, the weighted mean yields superior predictive performance. This work provides the first systematic evidence of these properties and establishes an interpretable, quantifiable framework for affective computing in data visualization.

This study addresses the limited understanding of how emotion is integrated into the design process within affective visualization research. Through semi-structured interviews with 15 visualization practitioners and a mixed-methods thematic analysis, the work systematically examines the mechanisms by which emotion is incorporated in practice across three dimensions: data, design, and audience. The findings reveal, for the first time, that emotional intent often emerges organically during design and that emotional impact arises from the synergistic effect of holistic design decisions rather than the summation of individual visual elements. The paper articulates emotion’s functional roles in guiding entry points, fostering engagement, and shaping interpretive outcomes, distills key design strategies, and identifies the evaluation of emotional efficacy as a primary challenge in the field.

This study addresses the fragmented and subjectively driven nature of affective behavior design in animal-inspired social robots, which often lacks systematic support. To bridge this gap, the authors analyze human-pet interaction videos, synthesize findings from literature, and conduct user interviews to construct a structured set of affective behavior reference cards. They further develop MojiKit—an integrated toolkit comprising these data-driven design resources, a pet-inspired robotic prototype named MomoBot, and a no-code behavior authoring studio. As the first data-driven resource for pet-inspired affective interaction design, MojiKit was evaluated in a co-creation workshop with 18 participants, who generated 35 distinct affective interaction patterns that extended beyond their personal pet-owning experiences. The toolkit significantly lowers technical barriers while enhancing design diversity and creative autonomy.