QCS:Feature Refining from Quadruplet Cross Similarity for Facial Expression Recognition

2024-11-04

arXiv.org

AI Summary

Facial expression recognition (FER) suffers from entangled label-relevant and redundant features, as well as weak cross-image discriminability. Method: This paper proposes the Quadruple Central-Symmetric network (QCS), a novel four-branch architecture featuring Cross-Similarity Attention (CSA)—the first mechanism enabling fine-grained quadruplet-level similarity modeling across images. It further introduces contrastive residual distillation to enhance feature discriminability and adopts a training-time multi-branch collaborative paradigm with inference-time single-branch efficiency—eliminating gradient conflicts while incurring zero additional inference overhead. Contribution/Results: QCS achieves state-of-the-art performance on multiple FER benchmarks, significantly improving intra-class feature compactness and inter-class separability without architectural or computational penalties.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Feature Construction/Reformulation
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Facial expression recognition faces challenges where labeled significant features in datasets are mixed with unlabeled redundant ones. In this paper, we introduce Cross Similarity Attention (CSA) to mine richer intrinsic information from image pairs, overcoming a limitation when the Scaled Dot-Product Attention of ViT is directly applied to calculate the similarity between two different images. Based on CSA, we simultaneously minimize intra-class differences and maximize inter-class differences at the fine-grained feature level through interactions among multiple branches. Contrastive residual distillation is utilized to transfer the information learned in the cross module back to the base network. We ingeniously design a four-branch centrally symmetric network, named Quadruplet Cross Similarity (QCS), which alleviates gradient conflicts arising from the cross module and achieves balanced and stable training. It can adaptively extract discriminative features while isolating redundant ones. The cross-attention modules exist during training, and only one base branch is retained during inference, resulting in no increase in inference time. Extensive experiments show that our proposed method achieves state-of-the-art performance on several FER datasets.

Problem

• Facial Expression Recognition
• Similarity Enhancement
• Discriminability Improvement

Innovation

• Quadruple Cross Similarity (QCS)
• Cross Similarity Attention (CSA) mechanism
• Facial Expression Recognition