VERITAS: Verifier-Guided Proof Search for Zero-Shot Formal Theorem Proving

2026-06-17

AI Summary

This work addresses the underutilization of rich verifier feedback—such as syntax errors and type mismatches—in existing large language model–driven formal theorem proving, where such signals are typically reduced to binary pass/fail outcomes. To remedy this, the authors propose VERITAS, a framework that, for the first time, systematically leverages the full spectrum of verifier signals in a zero-shot setting to guide proof search. VERITAS operates in two stages: an initial Best-of-N sampling phase generates candidate proofs, followed by a critic-guided Monte Carlo Tree Search (MCTS) phase that explicitly repurposes failed attempts from the first stage as negative examples to drive feedback-informed exploration. This two-stage design ensures attributional transparency, guaranteeing that all newly discovered solutions are directly guided by verifier feedback. Experiments demonstrate that VERITAS achieves a 40.6% success rate on miniF2F, surpassing Best-of-5 (36.9%), and establishes a significant 7.3% performance lead on the new benchmark VERITAS-CombiBench.

Abstract

LLM-based formal provers often collapse rich verifier signals (syntax errors, type mismatches, partial goal progress) into a binary pass/fail bit. We present VERITAS, a zero-shot framework that routes every verifier signal back into proof search through a two-phase protocol: Best-of-N sampling first, then a critic-guided MCTS pass that ingests Phase 1 failures as explicit negative examples. The protocol preserves every theorem solved by its own Phase 1 sweep, so Phase 2's additional solves are attributable to feedback-driven exploration. VERITAS reaches 40.6% on miniF2F (vs. an independently run Best-of-5 at 36.9%, Portfolio 26.2%) and 7.3% on VERITAS-CombiBench, a 55-theorem combinatorics benchmark we release on which Best-of-5 (1.8%) falls below Portfolio (3.6%), exposing that unguided sampling hurts when correct lemma names must be recovered iteratively from verifier feedback. Artifacts are available on GitHub.

Problem

• formal theorem proving
• verifier feedback
• zero-shot reasoning
• proof search
• large language models

Innovation

• verifier-guided proof search
• zero-shot theorem proving
• Monte Carlo Tree Search (MCTS)
• formal verification feedback
• combinatorics benchmark