Lazy or Efficient? Towards Accessible Eye-Tracking Event Detection Using LLMs

2026-04-14

AI Summary

This study addresses the limitations of traditional eye movement event detection, which relies heavily on programming expertise and is sensitive to data preprocessing and parameter tuning, thereby hindering deployment in non-specialist settings. To overcome these barriers, this work proposes the first large language model (LLM)-based, zero-code framework for eye-tracking analysis. The system enables end-to-end analytical pipelines—from raw data parsing and cleaning to event labeling (e.g., fixations and saccades) and algorithm optimization—through natural language instructions. It automatically infers data structure and integrates established algorithms such as I-VT and I-DT. Evaluated on public benchmarks, the method achieves accuracy comparable to conventional approaches while substantially lowering technical barriers, thus enhancing the accessibility and interpretability of eye movement research.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Gaze event detection is fundamental to vision science, human-computer interaction, and applied analytics. However, current workflows often require specialized programming knowledge and careful handling of heterogeneous raw data formats. Classical detectors such as I-VT and I-DT are effective but highly sensitive to preprocessing and parameterization, limiting their usability outside specialized laboratories. This work introduces a code-free, large language model (LLM)-driven pipeline that converts natural language instructions into an end-to-end analysis. The system (1) inspects raw eye-tracking files to infer structure and metadata, (2) generates executable routines for data cleaning and detector implementation from concise user prompts, (3) applies the generated detector to label fixations and saccades, and (4) returns results and explanatory reports, and allows users to iteratively optimize their code by editing the prompt. Evaluated on public benchmarks, the approach achieves accuracy comparable to traditional methods while substantially reducing technical overhead. The framework lowers barriers to entry for eye-tracking research, providing a flexible and accessible alternative to code-intensive workflows.

Problem

• eye-tracking
• event detection
• accessibility
• usability
• heterogeneous data

Innovation

• LLM-driven pipeline
• code-free eye-tracking
• natural language interface
• gaze event detection
• automated data analysis