Improving Autoformalization Using Direct Dependency Retrieval

2025-11-14

AI Summary

Existing automated formalization methods for mathematical statements suffer from two key limitations: (1) insufficient context awareness, leading to hallucinated definitions and theorems; and (2) heavy reliance on retrieval modules with low precision and recall, hindering scalability to large-scale formal libraries. This paper proposes DDR (Direct Dependency Retrieval), a novel framework that abandons the conventional retrieve-then-rerank paradigm. Instead, DDR directly generates candidate dependencies from natural-language statements and efficiently verifies their existence in the formal library using suffix arrays. We fine-tune the DDR model on a newly constructed large-scale dependency retrieval dataset and integrate retrieval-augmented generation (RAG) for end-to-end formalization. Experiments demonstrate that DDR achieves significant gains over state-of-the-art methods on dependency retrieval; moreover, the DDR-driven formalization system attains higher accuracy and robustness—both in single- and multi-attempt settings—establishing a new paradigm for scalable, high-fidelity mathematical formalization.

Technology Category

• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The convergence of deep learning and formal mathematics has spurred research in formal verification. Statement autoformalization, a crucial first step in this process, aims to translate informal descriptions into machine-verifiable representations but remains a significant challenge. The core difficulty lies in the fact that existing methods often suffer from a lack of contextual awareness, leading to hallucination of formal definitions and theorems. Furthermore, current retrieval-augmented approaches exhibit poor precision and recall for formal library dependency retrieval, and lack the scalability to effectively leverage ever-growing public datasets. To bridge this gap, we propose a novel retrieval-augmented framework based on DDR ( extit{Direct Dependency Retrieval}) for statement autoformalization. Our DDR method directly generates candidate library dependencies from natural language mathematical descriptions and subsequently verifies their existence within the formal library via an efficient suffix array check. Leveraging this efficient search mechanism, we constructed a dependency retrieval dataset of over 500,000 samples and fine-tuned a high-precision DDR model. Experimental results demonstrate that our DDR model significantly outperforms SOTA methods in both retrieval precision and recall. Consequently, an autoformalizer equipped with DDR shows consistent performance advantages in both single-attempt accuracy and multi-attempt stability compared to models using traditional selection-based RAG methods.

Problem

• Autoformalization lacks contextual awareness causing definition hallucinations
• Current retrieval methods have poor precision for formal dependencies
• Existing approaches lack scalability with growing mathematical datasets

Innovation

• Direct Dependency Retrieval for autoformalization
• Generates candidate dependencies from natural language
• Verifies dependencies via efficient suffix array check