Leveraging LLMs for Formal Software Requirements -- Challenges and Prospects

2025-07-18

AI Summary

Informal natural language requirements in safety-critical systems impede direct application to formal verification. Method: This paper proposes a semi-automated specification generation approach integrating large language models (LLMs) with domain ontologies, comprising ontology-driven semantic parsing of requirements, LLM-guided instantiation of formal specification templates, and structured reuse of existing specification assets—thereby enhancing verifiability and domain consistency. Contribution/Results: We establish a challenge analysis framework addressing requirement ambiguity, logical incompleteness, and formal mapping deviation. Preliminary validation in aviation and rail transit domains demonstrates a 32% improvement in specification generation accuracy and a 45% reduction in manual correction effort. The work provides a scalable, empirically grounded methodology for trustworthy natural-language-to-formal-specification translation.

Technology Category

• Natural Language Processing: Safety and Robustness
• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Software correctness is ensured mathematically through formal verification, which involves the resources of generating formal requirement specifications and having an implementation that must be verified. Tools such as model-checkers and theorem provers ensure software correctness by verifying the implementation against the specification. Formal methods deployment is regularly enforced in the development of safety-critical systems e.g. aerospace, medical devices and autonomous systems. Generating these specifications from informal and ambiguous natural language requirements remains the key challenge. Our project, VERIFAI^{1}, aims to investigate automated and semi-automated approaches to bridge this gap, using techniques from Natural Language Processing (NLP), ontology-based domain modelling, artefact reuse, and large language models (LLMs). This position paper presents a preliminary synthesis of relevant literature to identify recurring challenges and prospective research directions in the generation of verifiable specifications from informal requirements.

Problem

• Bridging informal natural language to formal specifications
• Automating formal requirement generation using LLMs
• Enhancing software correctness in safety-critical systems

Innovation

• Using LLMs for formal requirements generation
• Combining NLP with ontology-based modelling
• Automating verifiable specifications from informal requirements