Multi-scale Object-Aware Gaze Estimation via Geometric Reasoning

2026-06-28

AI Summary

This work addresses the challenge of semantic inconsistency in existing gaze target estimation methods, which typically rely on pixel-level regression and struggle in complex scenes. The study reframes the task as an object-centric hierarchical reasoning problem and introduces a two-stage framework: the first stage identifies potential gazed objects, while the second stage achieves precise localization by integrating object semantics–guided feature alignment, multi-scale fusion, and geometric constraints derived from head pose and gaze direction. By explicitly modeling semantic entities and incorporating geometric priors, the proposed method achieves state-of-the-art performance with only 7.1M parameters, attaining AUC scores of 0.961, 0.948, 0.987, and 0.977 on the GazeFollow, VideoAttentionTarget, ChildPlay, and GOO-Real datasets, respectively, significantly improving both semantic consistency and localization accuracy.

Abstract

Gaze target estimation aims to predict the semantic object an observer fixates upon within an image, a task deeply rooted in the object-oriented nature of human gaze. Observers tend to select a specific semantic entity as the attentional target, rather than responding randomly across arbitrary regions of the image. However, existing methods typically model this task as a direct mapping from global features to gaze heatmaps, essentially treating it as a pixel-level regression problem. This approach fails to explicitly represent the gazed object as a distinct entity, making it difficult to produce stable and semantically consistent predictions in complex scenes. To address this, we propose a two-stage gaze estimation framework guided by object semantics, reformulating gaze target estimation as a hierarchical reasoning process. Our method incorporates object-level representations during feature encoding to align image features with discrete semantic entities, then introduces multi-scale feature fusion and geometric constraints from head pose and gaze direction for fine-grained localization and object-level discrimination. Extensive experiments on GazeFollow, VideoAttentionTarget, ChildPlay, and GOO-Real demonstrate that our method achieves AUC of 0.961, 0.948, 0.987, and 0.977 respectively, delivering strong performance across all benchmarks while maintaining a compact parameter size of 7.1M.

Problem

• gaze target estimation
• object-aware
• semantic consistency
• complex scenes
• pixel-level regression

Innovation

• object-aware gaze estimation
• geometric reasoning
• multi-scale feature fusion
• semantic object alignment
• two-stage hierarchical reasoning