A Perception CNN for Facial Expression Recognition

2025-12-06

AI Summary

Existing CNNs for facial expression recognition (FER) suffer from neglecting facial segmentation and exhibit limited robustness to subtle local variations, occlusions, and pose changes. To address these limitations, this paper proposes the Perception Convolutional Neural Network (PCNN), a multi-branch architecture comprising five parallel pathways that explicitly model key facial regions—eyes, cheeks, and mouth—while integrating local perceptual features with global facial structural cues via a multi-domain interaction mechanism. A two-stage loss function is introduced to jointly optimize expression classification accuracy and image reconstruction fidelity. The model is trained end-to-end in a fully supervised manner. Extensive experiments on benchmark datasets—including CK+, JAFFE, FER2013, FERPlus, and RAF-DB—as well as occlusion- and pose-varied subsets demonstrate state-of-the-art performance, significantly enhancing both robustness and accuracy of FER under challenging real-world conditions.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Convolutional neural networks (CNNs) can automatically learn data patterns to express face images for facial expression recognition (FER). However, they may ignore effect of facial segmentation of FER. In this paper, we propose a perception CNN for FER as well as PCNN. Firstly, PCNN can use five parallel networks to simultaneously learn local facial features based on eyes, cheeks and mouth to realize the sensitive capture of the subtle changes in FER. Secondly, we utilize a multi-domain interaction mechanism to register and fuse between local sense organ features and global facial structural features to better express face images for FER. Finally, we design a two-phase loss function to restrict accuracy of obtained sense information and reconstructed face images to guarantee performance of obtained PCNN in FER. Experimental results show that our PCNN achieves superior results on several lab and real-world FER benchmarks: CK+, JAFFE, FER2013, FERPlus, RAF-DB and Occlusion and Pose Variant Dataset. Its code is available at https://github.com/hellloxiaotian/PCNN.

Problem

• Develops a CNN for facial expression recognition using parallel networks.
• Integrates local and global facial features via multi-domain interaction.
• Enhances accuracy with a two-phase loss function for robust performance.

Innovation

• Parallel networks learn local facial features
• Multi-domain interaction fuses local and global features
• Two-phase loss function restricts accuracy and reconstruction