Agentic Proof and Property-Based Testing via Property-Templates in Data-Intensive Computing

2026-07-09

AI Summary

This work addresses the challenge of misalignment between user intent and implementation in AI-generated code, particularly the difficulty of ensuring attribute correctness in data-intensive systems. To tackle this issue, the authors propose a novel dual-track verification framework based on parameterized property templates. This framework uniquely integrates formal verification in Lean 4 with executable property-based testing in PySpark, enabling unified validation through co-guided proof and testing. Experimental results demonstrate that the approach improves proof success rates by 1.6×, reduces hallucination by 59%, decreases intent mismatches from 22 to 1, lowers synthesis cost by 3.8×, and achieves higher code coverage than existing Spark fuzzing tools.

Abstract

As the cost of code generation becomes cheaper with AI, the new bottleneck in software engineering has shifted to intent specification and validation. Overcoming this durability crisis of AI-driven coding requires more than traditional fuzzing: each candidate property must be proven correct over a model and shown to hold on the real implementation, making formal proof and systematic property-based testing (PBT) complementary. However, validating properties this way at scale requires solving two subproblems: verifying candidate properties and operationalizing PBT without AI hallucination. We hypothesize that recurring property patterns, cast as property templates--abstract, parameterized forms with holes--address both at once. This paper investigates recurring property patterns in Apache Spark. In data-intensive scalable computing systems, correctness properties arise from the principles of data partition, computation decomposition, and dataflow computation. For instance, aggregation decomposition relates a global function executed on the entire dataset to a local function followed by a recombiner. We design an agentic, dual-track validation framework that uses property templates to formally verify correctness in the Lean 4 theorem prover and instantiate PBT templates as executable PySpark tests. Our evaluation shows that property templates increase agentic proof engineering success by up to 2.6x (1.6x on average) and reduce proof hallucinations by 59%. Template-guided PBT synthesis reduces intent misalignments from 22 to 1 and cuts synthesis cost by up to 5.7x (3.8x on average). Template-guided synthesis further exceeds a state-of-the-art Spark fuzzer and approaches unguided LLM-based PBT on code coverage. Finally, comparing the two tracks is informative: when a proof succeeds yet a PBT finds a counterexample, the mismatch identifies a gap between the formal model and implementation.

Problem

• intent specification
• property validation
• AI hallucination
• property-based testing
• formal verification

Innovation

• property templates
• agentic proof
• property-based testing
• formal verification
• data-intensive computing