LLMs as Educational Analysts: Transforming Multimodal Data Traces into Actionable Reading Assessment Reports

2025-03-03

AI Summary

Current reading assessments lack actionable behavioral insights. Method: This study proposes a multimodal educational analytics framework that integrates heterogeneous data—including eye-tracking trajectories, learning outcomes, assessment content, and curriculum standards—and applies unsupervised clustering (K-means/HDBSCAN) to uncover latent reading behavior patterns. Crucially, large language models (LLaMA/GPT series) are employed as “educational analysts,” guided by prompt engineering to generate end-to-end, pedagogically grounded instructional reports. The framework emphasizes human-AI collaboration, balancing automation efficiency with educational credibility via dual expert–teacher evaluation. Contribution/Results: Experimental validation demonstrates high acceptance of the generated reports across clarity, accuracy, relevance, and instructional utility (mean ≥4.6/5). This work constitutes the first empirical validation of LLMs’ effectiveness and practical viability in pedagogy-informed educational analysis.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Humans and AI: Brain-Sensing and Analysis

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Reading assessments are essential for enhancing students' comprehension, yet many EdTech applications focus mainly on outcome-based metrics, providing limited insights into student behavior and cognition. This study investigates the use of multimodal data sources -- including eye-tracking data, learning outcomes, assessment content, and teaching standards -- to derive meaningful reading insights. We employ unsupervised learning techniques to identify distinct reading behavior patterns, and then a large language model (LLM) synthesizes the derived information into actionable reports for educators, streamlining the interpretation process. LLM experts and human educators evaluate these reports for clarity, accuracy, relevance, and pedagogical usefulness. Our findings indicate that LLMs can effectively function as educational analysts, turning diverse data into teacher-friendly insights that are well-received by educators. While promising for automating insight generation, human oversight remains crucial to ensure reliability and fairness. This research advances human-centered AI in education, connecting data-driven analytics with practical classroom applications.

Problem

• Transforms multimodal data into actionable reading insights
• Uses LLMs to synthesize data for educator-friendly reports
• Ensures human oversight for reliability and fairness

Innovation

• Uses multimodal data for reading insights
• Applies unsupervised learning for behavior patterns
• LLM synthesizes data into actionable reports