CriticLean: Critic-Guided Reinforcement Learning for Mathematical Formalization

2025-07-08

AI Summary

Current approaches to translating natural language mathematical statements into formal Lean 4 code suffer from insufficient semantic fidelity. Method: This paper introduces CriticLeanGPT—a novel framework that elevates the critic module from passive verification to an active learning core. It jointly trains a semantic-correctness-aware critic model via supervised fine-tuning and reinforcement learning. A fine-grained semantic evaluation mechanism is designed, and we release FineLeanCorpus—the first high-diversity, high-quality formalization dataset (285K samples)—alongside CriticLeanBench, a dedicated evaluation benchmark. Contribution/Results: Experiments demonstrate that CriticLeanGPT significantly outperforms state-of-the-art open- and closed-source models on formalization correctness discrimination. The framework establishes a new paradigm for semantically reliable formalization in automated theorem proving.

Technology Category

• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Translating natural language mathematical statements into formal, executable code is a fundamental challenge in automated theorem proving. While prior work has focused on generation and compilation success, little attention has been paid to the critic phase-the evaluation of whether generated formalizations truly capture the semantic intent of the original problem. In this paper, we introduce CriticLean, a novel critic-guided reinforcement learning framework that elevates the role of the critic from a passive validator to an active learning component. Specifically, first, we propose the CriticLeanGPT, trained via supervised fine-tuning and reinforcement learning, to rigorously assess the semantic fidelity of Lean 4 formalizations. Then, we introduce CriticLeanBench, a benchmark designed to measure models' ability to distinguish semantically correct from incorrect formalizations, and demonstrate that our trained CriticLeanGPT models can significantly outperform strong open- and closed-source baselines. Building on the CriticLean framework, we construct FineLeanCorpus, a dataset comprising over 285K problems that exhibits rich domain diversity, broad difficulty coverage, and high correctness based on human evaluation. Overall, our findings highlight that optimizing the critic phase is essential for producing reliable formalizations, and we hope our CriticLean will provide valuable insights for future advances in formal mathematical reasoning.

Problem

• Enhancing semantic fidelity in math formalization via critic-guided RL
• Evaluating correctness of Lean 4 formalizations against original intent
• Building diverse datasets for reliable automated theorem proving

Innovation

• Critic-guided reinforcement learning framework
• CriticLeanGPT for semantic fidelity assessment
• FineLeanCorpus dataset with 285K problems