Characterizing Personality from Eye-Tracking: The Role of Gaze and Its Absence in Interactive Search Environments

2026-01-13

AI Summary

This study investigates the identification of the Big Five personality traits from users’ eye-tracking behavior in an interactive museum environment. The work proposes a multimodal time-series model that, for the first time, treats periods of missing gaze data—intervals when eye tracking fails—as meaningful behavioral signals rather than noise. By integrating these missing segments with raw eye-movement data, the model captures the relationship between personality and visual search behavior while minimizing preprocessing to preserve naturalistic patterns. Ablation studies confirm that incorporating missing-data intervals significantly enhances discriminative power. Evaluated via five-fold cross-validation, the model achieves strong performance across all personality dimensions (Macro F1: 73.09%–77.69%), with the inclusion of missing-gaze information yielding a 10–15% improvement in both accuracy and F1 score.

Technology Category

• Computer Vision: Multi-modal Vision
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Machine Learning: Multimodal Learning

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Personality traits influence how individuals engage, behave, and make decisions during the information-seeking process. However, few studies have linked personality to observable search behaviors. This study aims to characterize personality traits through a multimodal time-series model that integrates eye-tracking data and gaze missingness-periods when the user's gaze is not captured. This approach is based on the idea that people often look away when they think, signaling disengagement or reflection. We conducted a user study with 25 participants, who used an interactive application on an iPad, allowing them to engage with digital artifacts from a museum. We rely on raw gaze data from an eye tracker, minimizing preprocessing so that behavioral patterns can be preserved without substantial data cleaning. From this perspective, we trained models to predict personality traits using gaze signals. Our results from a five-fold cross-validation study demonstrate strong predictive performance across all five dimensions: Neuroticism (Macro F1 = 77.69%), Conscientiousness (74.52%), Openness (77.52%), Agreeableness (73.09%), and Extraversion (76.69%). The ablation study examines whether the absence of gaze information affects the model performance, demonstrating that incorporating missingness improves multimodal time-series modeling. The full model, which integrates both time-series signals and missingness information, achieves 10-15% higher accuracy and macro F1 scores across all Big Five traits compared to the model without time-series signals and missingness. These findings provide evidence that personality can be inferred from search-related gaze behavior and demonstrate the value of incorporating missing gaze data into time-series multimodal modeling.

Problem

• personality traits
• eye-tracking
• gaze missingness
• interactive search
• multimodal time-series

Innovation

• eye-tracking
• gaze missingness
• multimodal time-series modeling
• personality prediction
• Big Five traits