Towards Language Model Guided TLA+ Proof Automation

2025-12-10

AI Summary

TLA+ formal verification faces challenges including high manual proof effort, strict syntax constraints, and complex hierarchical structure, hindering automation. To address these, this paper proposes an LLM-driven, hierarchically guided proof automation framework. Our method constrains the LLM to generate only standardized subgoal decompositions—rather than full proofs—to avoid TLA+ syntax errors; introduces a hierarchical proposition decomposition mechanism integrated with an LLM-Symbolic Prover co-architecture; and constructs, for the first time, a dual-source benchmark suite comprising 119 theorems drawn from distributed protocols and mathematical induction. Experimental results demonstrate that our approach significantly outperforms all baselines across all theorems, achieving substantial improvements in subgoal decomposition accuracy and verification success rate, while markedly reducing error rates.

Technology Category

• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Formal theorem proving with TLA+ provides rigorous guarantees for system specifications, but constructing proofs requires substantial expertise and effort. While large language models have shown promise in automating proofs for tactic-based theorem provers like Lean, applying these approaches directly to TLA+ faces significant challenges due to the unique hierarchical proof structure of the TLA+ proof system. We present a prompt-based approach that leverages LLMs to guide hierarchical decomposition of complex proof obligations into simpler sub-claims, while relying on symbolic provers for verification. Our key insight is to constrain LLMs to generate normalized claim decompositions rather than complete proofs, significantly reducing syntax errors. We also introduce a benchmark suite of 119 theorems adapted from (1) established mathematical collections and (2) inductive proofs of distributed protocols. Our approach consistently outperforms baseline methods across the benchmark suite.

Problem

• Automates TLA+ proof construction using LLMs
• Handles hierarchical proof structures via decomposition
• Reduces syntax errors with normalized claim generation

Innovation

• LLM-guided hierarchical decomposition of proof obligations
• Constrained generation of normalized claim decompositions
• Integration of symbolic provers for verification