Findings of MEGA: Maths Explanation with LLMs using the Socratic Method for Active Learning

2025-07-16

AI Summary

Students with mathematics anxiety often struggle to comprehend higher-order mathematical problems, hindering conceptual understanding and metacognitive development. Method: We propose MEGA—a novel LLM-based pedagogical intervention framework integrating Socratic questioning, chain-of-thought (CoT) reasoning, lightweight gamification, and real-time formative feedback—designed to foster active engagement rather than passive exposition or standard CoT. Unlike conventional approaches, MEGA employs guided inquiry to stimulate self-directed reasoning, coupled with progress visualization and immediate feedback to close the learning loop. Contribution/Results: Experiments using GPT-4o and Claude 3.5 Sonnet on the MATH benchmark show that 47.5% of students rated MEGA’s explanations for challenging problems as substantially superior to standard CoT (26.67%). This validates MEGA’s efficacy in enhancing complex problem representation and metacognitive regulation. MEGA establishes a scalable, empirically evaluable paradigm for leveraging LLMs to support active, anxiety-mitigated mathematics learning.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Analogy

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

This paper presents an intervention study on the effects of the combined methods of (1) the Socratic method, (2) Chain of Thought (CoT) reasoning, (3) simplified gamification and (4) formative feedback on university students' Maths learning driven by large language models (LLMs). We call our approach Mathematics Explanations through Games by AI LLMs (MEGA). Some students struggle with Maths and as a result avoid Math-related discipline or subjects despite the importance of Maths across many fields, including signal processing. Oftentimes, students' Maths difficulties stem from suboptimal pedagogy. We compared the MEGA method to the traditional step-by-step (CoT) method to ascertain which is better by using a within-group design after randomly assigning questions for the participants, who are university students. Samples (n=60) were randomly drawn from each of the two test sets of the Grade School Math 8K (GSM8K) and Mathematics Aptitude Test of Heuristics (MATH) datasets, based on the error margin of 11%, the confidence level of 90%, and a manageable number of samples for the student evaluators. These samples were used to evaluate two capable LLMs at length (Generative Pretrained Transformer 4o (GPT4o) and Claude 3.5 Sonnet) out of the initial six that were tested for capability. The results showed that students agree in more instances that the MEGA method is experienced as better for learning for both datasets. It is even much better than the CoT (47.5% compared to 26.67%) in the more difficult MATH dataset, indicating that MEGA is better at explaining difficult Maths problems.

Problem

• Enhancing Math learning using Socratic method and LLMs
• Comparing MEGA method with traditional step-by-step teaching
• Addressing student difficulties in Math through innovative pedagogy

Innovation

• Socratic method enhances Math learning with LLMs
• Chain of Thought reasoning simplifies complex problems
• Gamification and feedback improve student engagement