AA: A Multi-view Multimodal Dataset for Screen-based Gaze Estimation

2026-06-30

AI Summary

This study addresses the limited robustness of gaze estimation models under varying viewpoints and occlusions by introducing a large-scale, multi-view, multi-modal dataset. The dataset synchronously captures facial images from eight built-in screen cameras and two lateral-view cameras, paired with precise screen-space gaze targets. It is the first to integrate both screen-centered and side-view perspectives, enabling joint modeling of global and local visual cues. The release includes subject-independent evaluation splits and a standardized preprocessing pipeline. Leveraging this resource, the proposed method substantially improves model generalization in challenging scenarios involving complex viewpoints and occlusions, thereby establishing a solid foundation for reproducible research in screen-based gaze estimation.

Abstract

We present AA, a multi-view multimodal dataset for screen-based gaze estimation. The dataset captures synchronized facial observations from eight fixed screen-mounted cameras and two additional side-view cameras, paired with precise screen-space gaze targets collected under controlled fixation conditions. Each sample contains multi-view face observations together with structured facial region crops, enabling multimodal learning from both global and local visual cues. Unlike existing single-view gaze datasets, AA provides multi-view coverage from both screen-mounted and side-mounted perspectives, enabling more robust modeling under viewpoint variation and occlusion. The dataset includes subject-independent evaluation splits and a standardized data processing pipeline to support reproducible research in gaze estimation.

Problem

• gaze estimation
• multi-view
• multimodal
• dataset
• screen-based

Innovation

• multi-view
• multimodal
• gaze estimation
• screen-based
• occlusion robustness