🤖 AI Summary
To address poor generalization in facial expression recognition caused by scarce labeled data under challenging conditions such as makeup and occlusion, this paper proposes an Implicit Sparse-Style Batch Transformer architecture. Our method integrates a dimension-aware sparse attention mechanism within an encoder-decoder framework, focusing exclusively on principal-feature dimensions to drastically reduce bottleneck-layer parameters. By jointly optimizing feature selection and implicit style modeling, it enhances both semantic consistency and diversity of synthesized images. Evaluation on small-scale datasets of makeup- and occlusion-affected faces demonstrates that the generated samples effectively augment training data variability, boosting downstream expression recognition accuracy by 4.2% and significantly improving model robustness. The core contribution is the first introduction of an implicit sparse attention paradigm tailored to low-dimensional critical features—achieving a favorable trade-off among computational efficiency, interpretability, and generative quality.
📝 Abstract
A novel Transformer variation architecture is proposed in the implicit sparse style. Unlike "traditional" Transformers, instead of attention to sequential or batch entities in their entirety of whole dimensionality, in the proposed Batch Transformers, attention to the "important" dimensions (primary components) is implemented. In such a way, the "important" dimensions or feature selection allows for a significant reduction of the bottleneck size in the encoder-decoder ANN architectures. The proposed architecture is tested on the synthetic image generation for the face recognition task in the case of the makeup and occlusion data set, allowing for increased variability of the limited original data set.