3D-Prover: Diversity Driven Theorem Proving With Determinantal Point Processes

2024-10-14

arXiv.org

AI Summary

In formal theorem proving, the exponential explosion of the search space—caused by an excessive number of candidate proof strategies—leads to high semantic redundancy and frequent strategy execution failures. To address this, we propose a semantics-aware strategy filtering framework: first, learning semantic embeddings of proof strategies via synthetic data; second, introducing Determinantal Point Processes (DPPs) for the first time into strategy selection to jointly model success probability, execution efficiency, and semantic diversity. The framework is agnostic to the underlying strategy generator—e.g., LLM-based systems such as ReProver—and supports plug-and-play integration. Evaluated on miniF2F-valid/test, our method achieves state-of-the-art (SOTA) performance in both proof completion rate and strategy success rate, while simultaneously reducing average execution time and enhancing semantic diversity among selected strategies.

Technology Category

• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

A key challenge in automated formal reasoning is the intractable search space, which grows exponentially with the depth of the proof. This branching is caused by the large number of candidate proof tactics which can be applied to a given goal. Nonetheless, many of these tactics are semantically similar or lead to an execution error, wasting valuable resources in both cases. We address the problem of effectively pruning this search, using only synthetic data generated from previous proof attempts. We first demonstrate that it is possible to generate semantically aware tactic representations which capture the effect on the proving environment, likelihood of success and execution time. We then propose a novel filtering mechanism which leverages these representations to select semantically diverse and high quality tactics, using Determinantal Point Processes. Our approach, 3D-Prover, is designed to be general, and to augment any underlying tactic generator. We demonstrate the effectiveness of 3D-Prover on the miniF2F-valid and miniF2F-test benchmarks by augmenting the ReProver LLM. We show that our approach leads to an increase in the overall proof rate, as well as a significant improvement in the tactic success rate, execution time and diversity.

Problem

• Prunes intractable search space in automated theorem proving
• Filters semantically similar and error-prone proof tactics
• Enhances proof success rate and tactic diversity

Innovation

• Generates semantically aware tactic representations from synthetic data
• Uses Determinantal Point Processes to select diverse tactics
• Augments existing proving LLMs to improve proof success rates