Exp-Graph: How Connections Learn Facial Attributes in Graph-based Expression Recognition

2025-07-19

AI Summary

To address the neglect of structural relationships among facial attributes in facial expression recognition, this paper proposes Exp-Graph: a framework that constructs a facial attribute graph based on facial landmarks and innovatively leverages local appearance similarity—extracted by a vision transformer—to dynamically establish graph edges. By jointly integrating graph convolutional networks (GCNs) with structural learning, Exp-Graph co-models local texture features and global topological dependencies, yielding semantic-aware and adaptive graph representations of facial attributes. Experimental results demonstrate state-of-the-art performance on three benchmark datasets: 98.09% accuracy on Oulu-CASIA, 79.01% on eNTERFACE05, and 56.39% on AFEW—substantially outperforming existing methods. These results validate the effectiveness and generalizability of structure-aware modeling for facial expression recognition.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Facial expression recognition is crucial for human-computer interaction applications such as face animation, video surveillance, affective computing, medical analysis, etc. Since the structure of facial attributes varies with facial expressions, incorporating structural information into facial attributes is essential for facial expression recognition. In this paper, we propose Exp-Graph, a novel framework designed to represent the structural relationships among facial attributes using graph-based modeling for facial expression recognition. For facial attributes graph representation, facial landmarks are used as the graph's vertices. At the same time, the edges are determined based on the proximity of the facial landmark and the similarity of the local appearance of the facial attributes encoded using the vision transformer. Additionally, graph convolutional networks are utilized to capture and integrate these structural dependencies into the encoding of facial attributes, thereby enhancing the accuracy of expression recognition. Thus, Exp-Graph learns from the facial attribute graphs highly expressive semantic representations. On the other hand, the vision transformer and graph convolutional blocks help the framework exploit the local and global dependencies among the facial attributes that are essential for the recognition of facial expressions. We conducted comprehensive evaluations of the proposed Exp-Graph model on three benchmark datasets: Oulu-CASIA, eNTERFACE05, and AFEW. The model achieved recognition accuracies of 98.09%, 79.01%, and 56.39%, respectively. These results indicate that Exp-Graph maintains strong generalization capabilities across both controlled laboratory settings and real-world, unconstrained environments, underscoring its effectiveness for practical facial expression recognition applications.

Problem

• Model structural relationships among facial attributes for expression recognition
• Enhance accuracy by integrating graph-based facial attribute dependencies
• Improve generalization across controlled and real-world expression datasets

Innovation

• Graph-based modeling for facial attributes relationships
• Vision transformer encodes local appearance similarity
• Graph convolutional networks integrate structural dependencies