🤖 AI Summary
Facial behavior analysis faces significant weakly supervised learning challenges, including high annotation costs, reliance on domain experts, ambiguous intensity labeling, and expert bias. To address these issues, this paper presents a systematic survey of weakly supervised learning methods for facial expression recognition and action unit detection in real-world scenarios. We propose the first unified weak supervision taxonomy encompassing both categorical and dimensional labels, explicitly characterizing core challenges such as label ambiguity and intensity bias. Methodologically, we introduce an integrated framework combining label-noise-robust training, multiple-instance learning, bag-level ranking supervision, self-training, and consistency regularization. Extensive evaluation—standardized across 12+ benchmark datasets and 30+ baseline methods—demonstrates the effectiveness and generalizability of our approach. Our work advances the practical deployment of facial behavior models under limited or imperfect supervision.
📝 Abstract
In the recent years, there has been a shift in facial behavior analysis from the laboratory-controlled conditions to the challenging in-the-wild conditions due to the superior performance of deep learning based approaches for many real world applications.However, the performance of deep learning approaches relies on the amount of training data. One of the major problems with data acquisition is the requirement of annotations for large amount of training data. Labeling process of huge training data demands lot of human support with strong domain expertise for facial expressions or action units, which is difficult to obtain in real-time environments.Moreover, labeling process is highly vulnerable to ambiguity of expressions or action units, especially for intensities due to the bias induced by the domain experts. Therefore, there is an imperative need to address the problem of facial behavior analysis with weak annotations. In this paper, we provide a comprehensive review of weakly supervised learning (WSL) approaches for facial behavior analysis with both categorical as well as dimensional labels along with the challenges and potential research directions associated with it. First, we introduce various types of weak annotations in the context of facial behavior analysis and the corresponding challenges associated with it. We then systematically review the existing state-of-the-art approaches and provide a taxonomy of these approaches along with their insights and limitations. In addition, widely used data-sets in the reviewed literature and the performance of these approaches along with evaluation principles are summarized. Finally, we discuss the remaining challenges and opportunities along with the potential research directions in order to apply facial behavior analysis with weak labels in real life situations.
