🤖 AI Summary
The COVID-19 pandemic accelerated the adoption of online proctoring via platforms like Zoom, yet real-time multi-stream video monitoring suffers from low efficiency and poor detection of anomalous behaviors. To address this, we propose the first lightweight intelligent proctoring system tailored for Zoom-based examinations. Our method introduces a novel automatic face ground-truth labeling technique that fuses dynamic name tags (positioned at the top-left corner of each participant’s window) with enhanced OCR. We further design a streaming video processing framework integrating: (i) lightweight YOLOv5 for real-time face detection; (ii) FaceNet embeddings with cosine similarity for identity verification; and (iii) PaddleOCR with dynamic coordinate mapping for robust text localization. The system achieves 90.4% real-time face detection accuracy and 98.4% accuracy in post-exam anomaly detection—including off-screen behavior, face turning, and impersonation—while maintaining sub-300 ms latency on standard faculty PCs. Source code is publicly available.
📝 Abstract
Online exams via video conference software like Zoom have been adopted in many schools due to COVID-19. While it is convenient, it is challenging for teachers to supervise online exams from simultaneously displayed student Zoom windows. In this paper, we propose iExam, an intelligent online exam monitoring and analysis system that can not only use face detection to assist invigilators in real-time student identification, but also be able to detect common abnormal behaviors (including face disappearing, rotating faces, and replacing with a different person during the exams) via a face recognition-based post-exam video analysis. To build such a novel system in its first kind, we overcome three challenges. First, we discover a lightweight approach to capturing exam video streams and analyzing them in real time. Second, we utilize the left-corner names that are displayed on each student's Zoom window and propose an improved OCR (optical character recognition) technique to automatically gather the ground truth for the student faces with dynamic positions. Third, we perform several experimental comparisons and optimizations to efficiently shorten the training and testing time required on teachers' PC. Our evaluation shows that iExam achieves high accuracy, 90.4% for real-time face detection and 98.4% for post-exam face recognition, while maintaining acceptable runtime performance. We have made iExam's source code available at https://github.com/VPRLab/iExam.
