Towards Real-World Industrial-Scale Verification: LLM-Driven Theorem Proving on seL4

2026-02-09

AI Summary

This work proposes AutoReal, a lightweight and deployable local large language model (LLM)-based theorem prover that addresses the high cost and expert dependency of industrial-scale formal verification, as exemplified by seL4. Built upon a 7B-parameter open-source LLM, AutoReal integrates chain-of-thought proof training, project-specific context augmentation, and seamless integration with Isabelle/HOL. Evaluated on 660 critical theorems from the seL4 codebase, AutoReal achieves a proof success rate of 51.67%, substantially outperforming the prior state-of-the-art rate of 27.06%. It further demonstrates strong generalization by attaining a 53.88% success rate on 451 theorems across three security-critical projects in the Archive of Formal Proofs (AFP). The approach also enhances proof interpretability, offering a practical pathway toward accessible and scalable formal verification.

Technology Category

• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Formal methods (FM) are reliable but costly to apply, often requiring years of expert effort in industrial-scale projects such as seL4, especially for theorem proving. Recent advances in large language models (LLMs) have made automated theorem proving increasingly feasible. However, most prior work focuses on mathematics-oriented benchmarks such as miniF2F, with limited evaluation on real-world verification projects. The few studies that consider industrial-scale verification mostly rely on closed-source models with hundreds of billions of parameters, which cannot be locally deployed and incur substantial usage costs. In this paper, we propose AutoReal, an LLM-driven theorem proving method for real-world industrial-scale systems with support for lightweight local deployment. We evaluate AutoReal on the seL4-Isabelle verification project as a representative and challenging case study. AutoReal incorporates two key improvements: (1) chain-of-thought (CoT)-based proof training, which teaches the LLM the reasoning behind proof steps and enables step-wise explanations alongside proofs, and (2) context augmentation, which leverages proof context from the project to enhance LLM-driven proving. Based on the AutoReal methodology, we fine-tune a base model to obtain AutoReal-Prover, a compact 7B-scale prover for industrial-scale theorem proving. AutoReal-Prover achieves a 51.67% proof success rate on 660 theorems from seL4-designated Important Theories across all 10 seL4 proof categories, substantially outperforming prior attempts on seL4 (27.06%). To evaluate generalization, we further apply AutoReal-Prover to three security-related projects from the Archive of Formal Proofs (AFP), covering all 451 theorems and achieving a proof success rate of 53.88%. Overall, this work advances the application of LLM-driven theorem proving in real-world industrial-scale verification.

Problem

• industrial-scale verification
• LLM-driven theorem proving
• seL4
• formal methods
• local deployment

Innovation

• LLM-driven theorem proving
• industrial-scale verification
• chain-of-thought reasoning
• context augmentation
• local deployment