MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detection

2025-03-23

AI Summary

Accurately identifying errors in K–12 multimodal math assignments—comprising both handwritten or typeset text and diagrams—remains challenging, as current multimodal large language models (MLLMs) lack the capability to jointly reason over image and text modalities and precisely localize and attribute solution-step errors. Method: This paper proposes the first three-stage mathematical agent hybrid framework: (1) image-text consistency verification, (2) visual-semantic parsing, and (3) cross-modal error integration analysis—explicitly modeling multimodal associations between problem statements and solution steps. The framework integrates vision understanding, symbolic logical reasoning, and pedagogically grounded constraints within a specialized collaborative agent architecture. Contribution/Results: Evaluated on real-world educational data, the framework improves step-level error detection accuracy by 5% and error-type classification accuracy by 3%. It has been deployed at scale across a platform serving over one million students, achieving 90% user satisfaction and substantially reducing manual review overhead.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Mathematical error detection in educational settings presents a significant challenge for Multimodal Large Language Models (MLLMs), requiring a sophisticated understanding of both visual and textual mathematical content along with complex reasoning capabilities. Though effective in mathematical problem-solving, MLLMs often struggle with the nuanced task of identifying and categorizing student errors in multimodal mathematical contexts. Therefore, we introduce MathAgent, a novel Mixture-of-Math-Agent framework designed specifically to address these challenges. Our approach decomposes error detection into three phases, each handled by a specialized agent: an image-text consistency validator, a visual semantic interpreter, and an integrative error analyzer. This architecture enables more accurate processing of mathematical content by explicitly modeling relationships between multimodal problems and student solution steps. We evaluate MathAgent on real-world educational data, demonstrating approximately 5% higher accuracy in error step identification and 3% improvement in error categorization compared to baseline models. Besides, MathAgent has been successfully deployed in an educational platform that has served over one million K-12 students, achieving nearly 90% student satisfaction while generating significant cost savings by reducing manual error detection.

Problem

• Detect multimodal math errors in education using MLLMs
• Improve accuracy in identifying and categorizing student errors
• Deploy efficient error detection for K-12 educational platforms

Innovation

• Mixture-of-Math-Agent framework for error detection
• Specialized agents for multimodal content processing
• Deployed in educational platform with high accuracy