Eyes Can't Always Tell: Fusing Eye Tracking and User Priors for User Modeling under AI Advice Conditions

2026-04-02

AI Summary

This study addresses the limited robustness of conventional eye-tracking models in AI-assisted decision-making, which arises because the mapping between oculomotor signals and users’ cognitive states—such as cognitive load and decision confidence—is modulated by AI advice reliability and individual differences. Through an eye-tracking experiment, the authors systematically investigate this dynamic relationship under three conditions: no AI assistance, correct AI advice, and incorrect AI advice, collecting both behavioral and self-report data. They propose a personalized modeling paradigm that integrates eye-tracking features with user prior information, including demographics, AI literacy, and technology trust. Results demonstrate that accurate AI advice significantly reduces cognitive load and enhances decision confidence. Moreover, models relying solely on eye-tracking features exhibit poor cross-condition generalizability, whereas incorporating user priors substantially improves predictive performance.

Abstract

Modeling users' cognitive states (e.g., cognitive load and decision confidence) is essential for building adaptive AI in high-stakes decision-making. While eye tracking provides non-invasive behavioral signals correlated with cognitive effort, prior work has not systematically examined how AI assistance contexts, specifically varying advice reliability and user heterogeneity, can alter the mapping between gaze signals and cognitive states. We conducted a within-subject lab eye-tracking study (N=54) on factual verification tasks under three conditions: No-AI, Correct-AI advice, and Incorrect-AI advice. We analyze condition-dependent changes in self-reports and eye-tracking patterns and evaluate the robustness of eye-tracking-based user modeling. Results show that AI advice increases decision confidence compared to No-AI, while Correct-AI is associated with lower perceived cognitive load and more efficient gaze behavior. Crucially, predictive modeling is context-sensitive: the relationship between eye-tracking signals and cognitive states shifts across AI conditions. Finally, fusing eye-tracking features with user priors (demographics, AI literacy/experience, and propensity to trust technology) improves cross-participant generalization. These findings support condition-aware and personalized user modeling for cognitively aligned adaptive AI systems.

Problem

• user modeling
• eye tracking
• cognitive states
• AI advice
• user heterogeneity

Innovation

• eye tracking
• user modeling
• AI advice
• cognitive states
• user priors