P$^{3}$: Joint Program-and-Proof Planning for Verified Code Generation

2026-08-10

AI Summary

Existing large language models generate verifiable code through a sequential pipeline that separates program synthesis from proof construction, often yielding code that is difficult to verify or contains subtle bugs, necessitating costly and iterative repairs. Inspired by Dijkstra’s principle that programs and their correctness arguments should be co-developed, this work proposes the P³ framework, which introduces, for the first time, joint planning of programs and proofs: a unified plan is first derived from formal specifications, followed by synchronized generation of both implementation and proof. The contributions include a novel joint planning mechanism, the creation of Lean4Commit0—the first Lean4 verification benchmark grounded in real-world software libraries—and the integration of LLM agents, formal verification, and API-level specifications. P³ achieves state-of-the-art solve rates across Verina, AlgoVeri, and Lean4Commit0, outperforming the strongest baseline by 4.6–11.2 percentage points while reducing API invocation costs by ~40% and runtime by 37%.

Abstract

Verified code generation asks a large language model (LLM) to generate both an executable program and a machine-checkable proof that the program meets a formal specification, promising software that is correct by construction. The de facto workflow decouples the two halves of the problem: first synthesize a program, then attempt to prove it correct. We observe that this sequential pipeline can be both ineffective and inefficient in practice. A program generated without anticipating its proof can be subtly incorrect or structurally difficult to verify, forcing the LLM into brittle repair loops that alternate between patching the code and patching the proof. Inspired by Dijkstra's view that a program and its correctness argument should be developed hand in hand, we propose $P^3$, an LLM-based agentic workflow that first derives a unified program-and-proof plan from the specification, then elaborates the implementation and proof scaffold under this shared plan. To evaluate verified code generation in realistic settings, we further introduce Lean4Commit0, a repository-derived, library-level benchmark built by extracting core APIs from real-world software repositories and translating their requirements, including relational specifications across APIs, into Lean tasks. Using four frontier LLM backends, we evaluate $P^3$ on Verina, AlgoVeri, and our Lean4Commit0 benchmark, where it achieves the highest solve rate in every benchmark--model setting. Compared with the stronger baseline, it improves solve rates by 4.6--11.2 percentage points and reduces per-task API cost by up to roughly 40\% and wall-clock time by up to roughly 37\% on the difficult subset of each benchmark. A targeted ablation further shows gains of 3.3--8.3 points over implementation-only planning, isolating the benefit of planning the program and proof jointly.

Problem

• verified code generation
• program synthesis
• formal verification
• correctness proof
• LLM-based reasoning

Innovation

• joint program-proof planning
• verified code generation
• large language models
• formal verification
• Lean4