Supervised Contrastive Learning for Ordinal Engagement Measurement

2025-05-27

AI Summary

In virtual learning environments, automatic student engagement measurement faces two key challenges: severe class imbalance and the inherent ordinal nature of engagement levels (e.g., low/medium/high), which conventional classification methods fail to model due to their neglect of ordinal relationships. To address these issues, this paper proposes an ordinal classification framework based on supervised contrastive learning. It is the first work to introduce supervised contrastive learning into engagement modeling, explicitly encoding inter-level ordinal constraints. The framework integrates video-based behavioral and affective features, employs a sequential classifier encoder, and incorporates multi-strategy temporal data augmentation to enhance robustness. Experiments on the DAiSEE dataset demonstrate that the proposed method achieves state-of-the-art performance in both accuracy and ordinal ranking consistency (ORC), significantly outperforming mainstream classification and ordinal regression approaches.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Video Understanding & Activity Analysis
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Student engagement plays a crucial role in the successful delivery of educational programs. Automated engagement measurement helps instructors monitor student participation, identify disengagement, and adapt their teaching strategies to enhance learning outcomes effectively. This paper identifies two key challenges in this problem: class imbalance and incorporating order into engagement levels rather than treating it as mere categories. Then, a novel approach to video-based student engagement measurement in virtual learning environments is proposed that utilizes supervised contrastive learning for ordinal classification of engagement. Various affective and behavioral features are extracted from video samples and utilized to train ordinal classifiers within a supervised contrastive learning framework (with a sequential classifier as the encoder). A key step involves the application of diverse time-series data augmentation techniques to these feature vectors, enhancing model training. The effectiveness of the proposed method was evaluated using a publicly available dataset for engagement measurement, DAiSEE, containing videos of students who participated in virtual learning programs. The results demonstrate the robust ability of the proposed method for the classification of the engagement level. This approach promises a significant contribution to understanding and enhancing student engagement in virtual learning environments.

Problem

• Addressing class imbalance in student engagement measurement
• Incorporating ordinal relationships in engagement level classification
• Developing video-based engagement analysis for virtual learning environments

Innovation

• Supervised contrastive learning for ordinal classification
• Time-series data augmentation for feature vectors
• Sequential classifier as encoder in framework