Natural Language based Specification and Verification

2026-05-11

AI Summary

This work proposes an end-to-end, large language model (LLM)-driven framework that integrates natural language directly into the formal verification pipeline, addressing the longstanding challenge that existing formal verification methods rely on rigorously defined formal specifications and thus struggle to accommodate safety requirements expressed in natural language. The approach leverages an LLM to automatically translate natural language descriptions into formal safety specifications, which are then used to perform compositional verification of code implementations. By circumventing the traditional dependency on manually crafted formal specifications, the method demonstrates a novel pathway toward bridging informal requirements and rigorous verification. Preliminary experiments indicate its feasibility and potential for enhancing code safety, offering a promising direction for making formal verification more accessible and applicable to real-world software development practices.

Abstract

Recent frontier large language models (LLMs) have shown strong performance in identifying security vulnerabilities in large, mature open-source systems. As LLM-generated code becomes increasingly common, a natural goal is to prevent such models from producing vulnerable implementations in the first place. Formal verification offers a principled route to this objective, but existing verification pipelines typically require specifications written in rigid formal languages. Prior work has explored using LLMs to synthesize such specifications, with limited success. In this paper, we investigate a different approach: using LLMs both to generate specifications and to verify implementations compositionally when the specifications are expressed in natural language. Our preliminary results suggest that this approach is promising.

Problem

• Natural Language Specification
• Formal Verification
• Large Language Models
• Security Vulnerabilities
• Code Generation

Innovation

• natural language specifications
• compositional verification
• large language models
• formal verification
• security vulnerabilities