LLM-based Multimodal Feedback Produces Equivalent Learning and Better Student Perceptions than Educator Feedback

2026-01-21

AI Summary

This study addresses the persistent challenge of delivering timely, accurate, and multimodal instructional feedback at scale in online learning environments. To this end, it proposes the first real-time feedback system that dynamically integrates large language models (LLMs) with multimodal resources—including text, slides, and speech—to generate context-aware, personalized feedback. The system synthesizes structured explanations, retrieves relevant lecture slides, and provides AI-generated spoken commentary to support learners. Experimental results demonstrate that the system matches teacher-provided feedback in terms of learning outcomes and significantly outperforms it across multiple dimensions: clarity, specificity, conciseness, motivational impact, learner satisfaction, and reduction of cognitive load. These advantages effectively facilitate students’ iterative revision of open-ended responses.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Providing timely, targeted, and multimodal feedback helps students quickly correct errors, build deep understanding and stay motivated, yet making it at scale remains a challenge. This study introduces a real-time AI-facilitated multimodal feedback system that integrates structured textual explanations with dynamic multimedia resources, including the retrieved most relevant slide page references and streaming AI audio narration. In an online crowdsourcing experiment, we compared this system against fixed business-as-usual feedback by educators across three dimensions: (1) learning effectiveness, (2) learner engagement, (3) perceived feedback quality and value. Results showed that AI multimodal feedback achieved learning gains equivalent to original educator feedback while significantly outperforming it on perceived clarity, specificity, conciseness, motivation, satisfaction, and reducing cognitive load, with comparable correctness, trust, and acceptance. Process logs revealed distinct engagement patterns: for multiple-choice questions, educator feedback encouraged more submissions; for open-ended questions, AI-facilitated targeted suggestions lowered revision barriers and promoted iterative improvement. These findings highlight the potential of AI multimodal feedback to provide scalable, real-time, and context-aware support that both reduces instructor workload and enhances student experience.

Problem

• multimodal feedback
• scalable feedback
• AI in education
• student learning
• instructor workload

Innovation

• LLM-based feedback
• multimodal feedback
• real-time AI tutoring
• scalable educational support
• cognitive load reduction