Estimating Central, Peripheral, and Temporal Visual Contributions to Human Decision Making in Atari Games

2026-04-06

AI Summary

This study investigates the relative contributions of central vision, peripheral vision, and temporal information to human decision-making in dynamic visual environments. Leveraging the Atari-HEAD eye-tracking dataset, the authors propose the first systematic and controllable ablation framework to inversely infer the roles of these multimodal visual cues. They train action prediction networks under six distinct input conditions and integrate behavioral clustering analysis to identify distinct decision-making paradigms. Experimental results demonstrate that removing peripheral vision leads to a median drop in prediction accuracy of 35.27–43.90%, substantially exceeding the performance degradation caused by ablating gaze maps (2.11–2.76%) or historical states (1.52–15.51%). These findings underscore the dominant role of peripheral vision in human decision-making and reveal a diversity of underlying visual strategies.

Abstract

We study how different visual information sources contribute to human decision making in dynamic visual environments. Using Atari-HEAD, a large-scale Atari gameplay dataset with synchronized eye-tracking, we introduce a controlled ablation framework as a means to reverse-engineer the contribution of peripheral visual information, explicit gaze information in form of gaze maps, and past-state information from human behavior. We train action-prediction networks under six settings that selectively include or exclude these information sources. Across 20 games, peripheral information shows by far the strongest contribution, with median prediction-accuracy drops in the range of 35.27-43.90% when removed. Gaze information yields smaller drops of 2.11-2.76%, while past-state information shows a broader range of 1.52-15.51%, with the upper end likely more informative due to reduced peripheral-information leakage. To complement aggregate accuracies, we cluster states by true-action probabilities assigned by the different model configurations. This analysis identifies coarse behavioral regimes, including focus-dominated, periphery-dominated, and more contextual decision situations. These results suggest that human decision making in Atari depends strongly on information beyond the current focus of gaze, while the proposed framework provides a way to estimate such information-source contributions from behavior.

Problem

• visual contributions
• human decision making
• peripheral vision
• eye-tracking
• Atari games

Innovation

• peripheral vision
• eye-tracking
• action prediction
• information ablation
• human decision making