MathForm: Scaling Mathematical Autoformalization with Knowledge Retrieval and Verification-Guided Refinement

2026-08-14

AI Summary

This study addresses the challenges of parameter memorization dependency, insufficient library knowledge, and inadequate feedback mechanisms in automatic mathematical formalization. We propose an iterative optimization framework integrating Mathlib retrieval with compiler-driven feedback. Leveraging the newly constructed FormalVerse dataset comprising 367K samples, we train an 8B model via supervised fine-tuning and reinforcement learning, introducing a novel retrieval planner and verification-guided refinement mechanism to overcome single-pass generation limitations. The resulting model achieves Pass@8 scores of 88.06% (SC) and 72.37% (CC) across six benchmarks, significantly outperforming multiple specialized 32B models. These results demonstrate substantial improvements in both syntactic correctness and semantic consistency for mathematical formalization tasks.

Abstract

Autoformalization is commonly framed as translating natural-language mathematical statements into machine-verifiable formal languages such as Lean 4. However, faithful formalization requires more than translation. Models must map mathematical concepts to the complex hierarchy of types and definitions in formal libraries such as Mathlib, while ensuring that generated statements preserve the meaning of the source propositions. Existing approaches struggle because they rely heavily on the model's parametric memory for library-specific knowledge, while common data construction pipelines often resort to filtering single-pass outputs and lack mechanisms for feedback-driven revision. To address these challenges, we introduce MathForm, an autoformalization framework for constructing verified training data through Mathlib knowledge retrieval and verification-guided iterative refinement. Before generation, a retrieval planner gathers relevant definitions and existing formalizations from Mathlib to guide the formalization generator. Generated statements are then revised using compiler diagnostics and semantic-consistency feedback. Using this framework, we construct FormalVerse, a Lean 4 dataset containing approximately 367K verified examples across diverse mathematical domains and sources. We then train MathForm-8B through supervised fine-tuning followed by reinforcement learning. Across six benchmarks, MathForm-8B achieves average Pass@8 rates of 88.06% under Syntax Check (SC) and 72.37% under Consistency Check (CC), outperforming multiple specialized 32B autoformalizers. On the challenging FATE-H and FATE-X subsets, it attains CC pass rates of 63% and 37%, exceeding the strongest specialized baselines in both cases.

Problem

• Autoformalization
• Semantic Consistency
• Formal Library Alignment
• Feedback-driven Revision

Innovation

• Autoformalization
• Knowledge Retrieval
• Verification-Guided Refinement
• Reinforcement Learning
• FormalVerse