Disentangling Emotional Bases and Transient Fluctuations: A Low-Rank Sparse Decomposition Approach for Video Affective Analysis

2025-11-14

AI Summary

In video emotion analysis, complex emotional dynamics often cause model instability and representation degradation, primarily due to the absence of a hierarchical disentanglement mechanism separating long-term affective baselines (stable tonal foundations) from short-term transient fluctuations (dynamic variations). To address this, we propose the first hierarchical emotion modeling framework based on low-rank sparse decomposition. It comprises three plug-and-play components: a Stability Encoding Module (SEM), a Dynamic Disentanglement Module (DDM), and a Consistency Integration Module (CIM), enabling explicit separation and collaborative reconstruction of emotion constituents. We further introduce Rank-Aware optimization and multi-scale feature reconstruction to enhance training stability and discriminability. Extensive experiments on multiple benchmark datasets demonstrate significant improvements in robustness and dynamic emotion recognition accuracy, validating the effectiveness and generalizability of hierarchical low-rank sparse modeling for video-based affective computing.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Cognitive Modeling & Cognitive Systems: Affective Computing
• Machine Learning: Multimodal Learning

Application Category

• User Modeling, Personalization and Recommendation: Personalized content ranking and presentation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Video-based Affective Computing (VAC), vital for emotion analysis and human-computer interaction, suffers from model instability and representational degradation due to complex emotional dynamics. Since the meaning of different emotional fluctuations may differ under different emotional contexts, the core limitation is the lack of a hierarchical structural mechanism to disentangle distinct affective components, i.e., emotional bases (the long-term emotional tone), and transient fluctuations (the short-term emotional fluctuations). To address this, we propose the Low-Rank Sparse Emotion Understanding Framework (LSEF), a unified model grounded in the Low-Rank Sparse Principle, which theoretically reframes affective dynamics as a hierarchical low-rank sparse compositional process. LSEF employs three plug-and-play modules, i.e., the Stability Encoding Module (SEM) captures low-rank emotional bases; the Dynamic Decoupling Module (DDM) isolates sparse transient signals; and the Consistency Integration Module (CIM) reconstructs multi-scale stability and reactivity coherence. This framework is optimized by a Rank Aware Optimization (RAO) strategy that adaptively balances gradient smoothness and sensitivity. Extensive experiments across multiple datasets confirm that LSEF significantly enhances robustness and dynamic discrimination, which further validates the effectiveness and generality of hierarchical low-rank sparse modeling for understanding affective dynamics.

Problem

• Separates long-term emotional bases from short-term fluctuations
• Addresses model instability in video affective computing
• Enhances dynamic emotion discrimination through hierarchical decomposition

Innovation

• Low-rank sparse decomposition disentangles emotional bases and fluctuations
• Plug-and-play modules capture stability and dynamic signals
• Rank aware optimization balances gradient smoothness and sensitivity