D'ej`a Vu? Decoding Repeated Reading from Eye Movements

2025-02-16

AI Summary

This paper introduces and formally models “repeated-reading discrimination” — a novel eye-movement decoding task that automatically determines whether a reader has previously encountered a given text, based solely on oculomotor behavior. Methodologically, it integrates cognition-inspired feature engineering (e.g., fixation duration, regression frequency) with LSTM and Transformer neural architectures, and pioneers the use of E-Z Reader cognitive model simulations to generate synthetic eye-tracking data for training augmentation. Crucially, the trained discriminator is repurposed as an interpretable tool for memory mechanism analysis. Evaluated across multiple heterogeneous eye-tracking datasets, the approach significantly outperforms established baselines. Interpretability analyses confirm the existence of stable, cross-participant, generalizable oculomotor signatures of prior exposure. This work establishes a new paradigm and provides a deployable framework for educational technology, adaptive reading systems, and cognitive neuroscience.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Be it your favorite novel, a newswire article, a cooking recipe or an academic paper -- in many daily situations we read the same text more than once. In this work, we ask whether it is possible to automatically determine whether the reader has previously encountered a text based on their eye movement patterns. We introduce two variants of this task and address them with considerable success using both feature-based and neural models. We further introduce a general strategy for enhancing these models with machine generated simulations of eye movements from a cognitive model. Finally, we present an analysis of model performance which on the one hand yields insights on the information used by the models, and on the other hand leverages predictive modeling as an analytic tool for better characterization of the role of memory in repeated reading. Our work advances the understanding of the extent and manner in which eye movements in reading capture memory effects from prior text exposure, and paves the way for future applications that involve predictive modeling of repeated reading.

Problem

• Predicting repeated text reading from eye movements.
• Enhancing models with simulated eye movement data.
• Analyzing memory's role in repeated reading patterns.

Innovation

• Feature-based models for analysis
• Neural models enhance detection
• Cognitive simulations improve predictions