Learning to Repair Lean Proofs from Compiler Feedback

2026-02-03

AI Summary

Existing Lean proof datasets lack erroneous examples and corresponding repair supervision signals, making it difficult to train models capable of understanding and correcting errors based on compiler feedback. This work formulates proof repair as a supervised learning task and introduces APRIL, the first large-scale repair dataset, constructed through systematically generated incorrect proofs, their associated compiler diagnostics, and the corresponding repairs paired with natural language explanations. By incorporating diagnostic-conditioned supervision signals, APRIL enables more effective learning of error diagnosis and correction. A fine-tuned 4B-parameter language model trained on APRIL significantly outperforms the strongest open-source baseline in single-attempt repair tasks, demonstrating the dataset’s effectiveness in enhancing both repair accuracy and the model’s ability to reason about compiler feedback.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

As neural theorem provers become increasingly agentic, the ability to interpret and act on compiler feedback is critical. However, existing Lean datasets consist almost exclusively of correct proofs, offering little supervision for understanding and repairing failures. We study Lean proof repair as a supervised learning problem: given an erroneous proof and compiler feedback, predict both a corrected proof and a natural-language diagnosis grounded in the same feedback. We introduce APRIL (Automated Proof Repair in Lean), a dataset of 260,000 supervised tuples pairing systematically generated proof failures with compiler diagnostics and aligned repair and explanation targets. Training language models on APRIL substantially improves repair accuracy and feedback-conditioned reasoning; in our single-shot repair evaluation setting, a finetuned 4B-parameter model outperforms the strongest open-source baseline. We view diagnostic-conditioned supervision as a complementary training signal for feedback-using provers. Our dataset is available at \href{https://huggingface.co/datasets/uw-math-ai/APRIL}{this link}.

Problem

• proof repair
• compiler feedback
• Lean
• neural theorem proving
• supervised learning

Innovation

• proof repair
• compiler feedback
• supervised learning
• Lean theorem proving
• diagnostic-conditioned supervision