🤖 AI Summary
This work addresses the challenge of deploying vision Transformer-based facial emotion recognition models on edge devices, where their O(N²) computational complexity poses significant limitations. To this end, we propose Sparse Attention to Emotion (SAE), the first approach to incorporate sparse attention mechanisms into this task. SAE dynamically prunes image tokens irrelevant to emotion discrimination, retaining only those from critical regions such as the eyes and mouth. Experimental results demonstrate that SAE achieves a new state-of-the-art accuracy on the RAF-DB dataset using approximately 10% of the original tokens, while reducing computational complexity by up to 90%. This substantial efficiency gain markedly enhances the model’s practicality for real-world deployment without compromising performance.
📝 Abstract
Facial Emotion Recognition (FER) is an important task that has significant implications across various fields such as biometrics, health, and human-computer interaction. Current Vision Transformer-based approaches display quadratic complexity $\mathcal{O}(N^2)$, with N being the input sequence length, making them cumbersome to deploy at the edge. In this paper, we hypothesize that the FER task does not necessarily require all facial information to correctly interpret emotional states, as specific regions such as the eyes, the mouth, and parts of the cheeks carry discriminative information that can be sufficient to recognize emotions. Based on this, we propose Sparse Attention to Emotion (SAE), a model that discards image tokens that have no added value to the emotional context, while preserving good accuracy and achieving a significant gain in computational cost. Surprisingly, even after suppressing 90\% of the image tokens, our model achieves competitive accuracy to state of the art methods at much lower cost, providing a lightweight Facial Emotion Recognition approach. Experimental results demonstrate that SAE achieves new state of the art results on the RAF-DB dataset while reducing the computational complexity by up to 90\%.