Wrong Face, Wrong Move: The Social Dynamics of Emotion Misperception in Agent-Based Models

2025-08-26

AI Summary

This study investigates the systemic impact of emotion perception accuracy on multi-agent social behavior. We trained a facial emotion classifier on the KDEF, JAFFE, and CK+ datasets and integrated it into emotion-driven agent models simulated on a 2D toroidal lattice, where agents engage in local interaction and attraction–repulsion dynamics. Results show that high-perception-accuracy classifiers foster stable emotional clustering and robustness against perturbations; conversely, low-accuracy perception triggers cascading sadness propagation, trust erosion, group segregation, and collapse of social cohesion—even from initially neutral configurations, inducing significant structural fragmentation. Our key contribution is the first quantitative demonstration of how perceptual misclassification initiates affective collapse and socio-organizational disorder via a “perception–trust–structure” cascade failure mechanism. This work advances theoretical foundations for AI-driven social modeling and empathic human–AI interaction design.

Technology Category

• Multiagent Systems: Agent-Based Simulation and Emergent Behavior
• Cognitive Modeling & Cognitive Systems: Affective Computing
• Humans and AI: Emotional Intelligence

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The ability of humans to detect and respond to others' emotions is fundamental to understanding social behavior. Here, agents are instantiated with emotion classifiers of varying accuracy to study the impact of perceptual accuracy on emergent emotional and spatial behavior. Agents are visually represented with face photos from the KDEF database and endowed with one of three classifiers trained on the JAFFE (poor), CK+ (medium), or KDEF (high) datasets. Agents communicate locally on a 2D toroidal lattice, perceiving neighbors' emotional state based on their classifier and responding with movement toward perceived positive emotions and away from perceived negative emotions. Note that the agents respond to perceived, instead of ground-truth, emotions, introducing systematic misperception and frustration. A battery of experiments is carried out on homogeneous and heterogeneous populations and scenarios with repeated emotional shocks. Results show that low-accuracy classifiers on the part of the agent reliably result in diminished trust, emotional disintegration into sadness, and disordered social organization. By contrast, the agent that develops high accuracy develops hardy emotional clusters and resilience to emotional disruptions. Even in emotionally neutral scenarios, misperception is enough to generate segregation and disintegration of cohesion. These findings underscore the fact that biases or imprecision in emotion recognition may significantly warp social processes and disrupt emotional integration.

Problem

• Studying emotion misperception impact on agent social behavior
• Analyzing classifier accuracy effects on emotional clustering dynamics
• Investigating misperception-induced segregation in social organization

Innovation

• Agent-based models with emotion classifiers
• Visual representation using KDEF face photos
• Movement based on perceived emotions