Which Feedback Works for Whom? Differential Effects of LLM-Generated Feedback Elements Across Learner Profiles

2026-02-12

AI Summary

This study investigates how to optimize feedback generated by large language models (LLMs) based on learners’ Big Five personality traits to enhance both learning outcomes and feedback acceptance. Leveraging six types of feedback elements, the authors employed GPT-5 to generate structured feedback for high school biology multiple-choice questions. A controlled experiment involving 321 ninth-grade students, combined with questionnaire data and personality-based clustering analysis, was conducted to evaluate differences in learning effectiveness and subjective preferences across feedback types. The findings reveal that feedback elements supporting learning outcomes demonstrate general efficacy, whereas learners’ subjective preferences for specific feedback types are significantly moderated by their personality traits. This work provides empirical evidence and a personality-aware LLM adaptation strategy for personalized educational feedback.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Learning Human Values and Preferences
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language models (LLMs) show promise for automatically generating feedback in education settings. However, it remains unclear how specific feedback elements, such as tone and information coverage, contribute to learning outcomes and learner acceptance, particularly across learners with different personality traits. In this study, we define six feedback elements and generate feedback for multiple-choice biology questions using GPT-5. We conduct a learning experiment with 321 first-year high school students and evaluate feedback effectiveness using two learning outcomes measures and subjective evaluations across six criteria. We further analyze differences in how feedback acceptance varies across learners based on Big Five personality traits. Our results show that effective feedback elements share common patterns supporting learning outcomes, while learners'subjective preferences differ across personality-based clusters. These findings highlight the importance of selecting and adapting feedback elements according to learners'personality traits when we design LLM-generated feedback, and provide practical implications for personalized feedback design in education.

Problem

• LLM-generated feedback
• learner profiles
• personality traits
• feedback effectiveness
• personalized feedback

Innovation

• LLM-generated feedback
• personalized feedback
• learner personality
• feedback elements
• educational AI