🤖 AI Summary
This work addresses the high cost of pixel-level annotation in medical image segmentation by proposing TrailNet, a novel model that leverages clinicians’ eye-tracking trajectories as the sole weak supervision signal. TrailNet jointly models the temporal dynamics of gaze patterns and spatial image semantics through a trajectory-guided spatiotemporal encoder that integrates fixation points with their contextual trajectories. To enhance robustness and suppress gaze-related noise, the method incorporates a multi-scale uncertainty-aware decoder, class-exclusive constraints, and a cyclic knowledge distillation mechanism. Notably, the trained model operates without requiring eye-tracking hardware during inference. Evaluated on two public datasets, TrailNet achieves Dice scores of 81.25% and 81.85%, respectively, significantly outperforming current state-of-the-art approaches.
📝 Abstract
Medical image segmentation heavily depends on labor-intensive and time-consuming pixel-level annotations. Eye tracking offers a cost-effective solution that can be naturally integrated into clinical workflows. Recorded by eye trackers, gaze conveys the spatial regions of clinicians' attention through fixations and the temporal context of clinicians' progressive visual perception from trajectories. Nevertheless, effective modeling of temporal trajectories remains challenging, and noise in gaze caused by exploratory fixations greatly limits segmentation performance. To overcome these limitations, we propose the Trajectory-guided Uncertainty-aware Network (TrailNet), which exploits gaze-supervised medical image segmentation from spatial semantics modeling to temporal context by jointly leveraging fixations and trajectories. Specifically, the proposed trajectory-guided spatio-temporal encoder models temporal context and establishes complementary interactions with image spatial semantics to strengthen target perception. Furthermore, the multi-scale uncertainty decoder leverages category mutual-exclusivity constraints to produce deterministic predictions and mitigate supervision uncertainty induced by noise. To enable gaze-free inference, we further introduce a cycle distillation strategy that transfers feature-level knowledge via teacher-student networks. Experimental results on two public datasets demonstrate that TrailNet outperforms state-of-the-art methods, achieving Dice scores of 81.25% and 81.85%, respectively.