AxQM: A Textbook-Scale Benchmark for Formal Proof Synthesis in a Library of Finite-Dimensional Quantum Mechanics
为解决物理领域理论论证的严谨性问题,通过构建AxQM基准测试集,使用Lean证明助手自动生成教科书规模的量子力学形式化证明。
为解决物理领域理论论证的严谨性问题,通过构建AxQM基准测试集,使用Lean证明助手自动生成教科书规模的量子力学形式化证明。
This work addresses the challenge of simultaneously generating executable code and formal verification artifacts—such as invariants, assertions, and termination proofs—in Dafny. It proposes a verifier-guided repair framework that, for the first time, deeply integrates agent-driven code generation with Dafny’s formal verification capabilities. By establishing an iterative feedback loop between a large language model and the Dafny verifier, the approach co-generates program implementations and their accompanying verification constructs. The study also introduces LCB-Pro-Dafny, the first Dafny verification benchmark tailored to competition-level problems. Experimental results demonstrate that the method achieves a 92.7% verification success rate on DafnyBench, surpassing the previous state-of-the-art baseline by 6.5 percentage points, while further revealing an orthogonality between verification success and runtime test performance.
This work addresses the lack of concise and reproducible baseline systems in AI-driven automated theorem proving, which hinders fair architectural comparisons. To this end, we propose a minimalist yet competitive proof agent that integrates three core mechanisms: iterative proof refinement, theorem library retrieval, and context management. The system enables systematic evaluation of diverse large language models and design choices, achieving performance on par with state-of-the-art methods across multiple heterogeneous benchmarks. Our experiments demonstrate that iterative proof generation significantly outperforms single-pass generation, offering superior sample efficiency and reduced inference cost. The codebase is publicly released to provide the community with a standardized reference implementation.
为解决物理领域理论论证的严谨性问题,通过构建AxQM基准测试集,使用Lean证明助手自动生成教科书规模的量子力学形式化证明。
This work addresses the challenge of simultaneously generating executable code and formal verification artifacts—such as invariants, assertions, and termination proofs—in Dafny. It proposes a verifier-guided repair framework that, for the first time, deeply integrates agent-driven code generation with Dafny’s formal verification capabilities. By establishing an iterative feedback loop between a large language model and the Dafny verifier, the approach co-generates program implementations and their accompanying verification constructs. The study also introduces LCB-Pro-Dafny, the first Dafny verification benchmark tailored to competition-level problems. Experimental results demonstrate that the method achieves a 92.7% verification success rate on DafnyBench, surpassing the previous state-of-the-art baseline by 6.5 percentage points, while further revealing an orthogonality between verification success and runtime test performance.
This work addresses the lack of concise and reproducible baseline systems in AI-driven automated theorem proving, which hinders fair architectural comparisons. To this end, we propose a minimalist yet competitive proof agent that integrates three core mechanisms: iterative proof refinement, theorem library retrieval, and context management. The system enables systematic evaluation of diverse large language models and design choices, achieving performance on par with state-of-the-art methods across multiple heterogeneous benchmarks. Our experiments demonstrate that iterative proof generation significantly outperforms single-pass generation, offering superior sample efficiency and reduced inference cost. The codebase is publicly released to provide the community with a standardized reference implementation.