Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges

2026-05-19

AI Summary

This work systematically evaluates and advances the robustness, generalization, and reasoning faithfulness of large language models in mathematical reasoning. Drawing on a comprehensive review of approximately 120 studies, it introduces a unified taxonomy for benchmark datasets and constructs an analytical framework encompassing model architectures, training strategies, and evaluation methodologies. The study innovatively uncovers a critical evaluation gap between process-level reasoning verification and final answer accuracy, identifying key challenges such as unfaithful reasoning, evaluation bias, and insufficient generalization. To address these issues, it advocates promising technical directions—including tool integration, verifier-guided reasoning, and parameter-efficient fine-tuning—thereby charting a path toward building reliable and trustworthy mathematical reasoning systems.

Abstract

Mathematical reasoning is essential for problem-solving in education, science, and industry, serving as a crucial benchmark for evaluating artificial intelligence systems. As Large Language Models (LLMs) improve their reasoning capabilities, understanding how well they perform mathematical reasoning has become increasingly important. This survey synthesizes recent advancements in mathematical reasoning with LLMs through a structured analysis of datasets, architectures, training strategies, and evaluation protocols. Our systematic review encompasses approximately 120 peer-reviewed studies and preprints, examining the evolution of this research area and providing a unified analytical framework to understand current progress and limitations. Our study particularly introduces a unified taxonomy of mathematical datasets, distinguishing between pretraining corpora, supervised fine-tuning resources, and evaluation benchmarks across varying levels of reasoning complexity. A systematic analysis of reasoning architectures and training strategies, including tool integration, verifier-guided reasoning, and parameter-efficient adaptation, is presented to assess their effects on reasoning robustness and generalization. Moreover, a comparative evaluation of existing metrics highlights the gap between final-answer accuracy and process-level reasoning verification. By synthesizing insights across these areas, our analysis identifies recurring failure modes, such as reasoning faithfulness issues, benchmark biases, and generalization limitations, and outlines key research directions toward improving symbolic grounding, evaluation reliability, and the development of more robust and trustworthy LLM-based reasoning systems.

Problem

• mathematical reasoning
• large language models
• evaluation benchmarks
• reasoning robustness
• generalization

Innovation

• mathematical reasoning
• large language models
• reasoning architectures
• evaluation metrics
• tool-augmented reasoning