S3c-Math: Spontaneous Step-level Self-correction Makes Large Language Models Better Mathematical Reasoners

2024-09-03

arXiv.org

AI Summary

This work addresses the lack of intrinsic, fine-grained self-correction capability in large language models (LLMs) for mathematical reasoning. To this end, we propose the S³c-Math series, the first LLMs capable of spontaneous, step-level self-correction during inference—without external intervention or post-hoc refinement. Methodologically, we introduce a step-level sampling–based self-correction data construction paradigm, coupled with targeted fine-tuning, enabling the model to perform internal validation and correction concurrently with each reasoning step. Our core contribution lies in establishing an end-to-end, endogenous step-level correction mechanism—breaking away from conventional reliance on retrospective correction, multi-model collaboration, or external knowledge sources. Evaluated on GSM8K and MATH benchmarks, S³c-Math achieves significant performance gains; moreover, the self-correction capability generalizes across diverse foundational LLM architectures.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Self-correction is a novel method that can stimulate the potential reasoning abilities of large language models (LLMs). It involves detecting and correcting errors during the inference process when LLMs solve reasoning problems. However, recent works do not regard self-correction as a spontaneous and intrinsic capability of LLMs. Instead, such correction is achieved through post-hoc generation, external knowledge introduction, multi-model collaboration, and similar techniques. In this paper, we propose a series of mathematical LLMs called S$^3$c-Math, which are able to perform Spontaneous Step-level Self-correction for Mathematical reasoning. This capability helps LLMs to recognize whether their ongoing inference tends to contain errors and simultaneously correct these errors to produce a more reliable response. We proposed a method, which employs a step-level sampling approach to construct step-wise self-correction data for achieving such ability. Additionally, we implement a training strategy that uses above constructed data to equip LLMs with spontaneous step-level self-correction capacities. Our data and methods have been demonstrated to be effective across various foundation LLMs, consistently showing significant progress in evaluations on GSM8K, MATH, and other mathematical benchmarks. To the best of our knowledge, we are the first to introduce the spontaneous step-level self-correction ability of LLMs in mathematical reasoning.

Problem

• Enhance LLMs' mathematical reasoning
• Implement spontaneous step-level self-correction
• Improve error detection and correction

Innovation

• Spontaneous Step-level Self-correction
• Step-wise Self-correction Data
• Training Strategy for Self-correction