From Natural Language to Verified Code: Toward AI Assisted Problem-to-Code Generation with Dafny-Based Formal Verification

2026-04-24

AI Summary

This work addresses the challenges of ensuring correctness and accurately constructing formal specifications when large language models generate code from natural language. We propose a verifiable code generation approach that integrates hierarchical prompting with verification feedback. To support this, we introduce the NL2VC-60 dataset, which leverages Dafny formal specifications and the uDebug platform to prevent vacuous verification. Furthermore, we design a self-repair prompting mechanism guided by structural signatures and verifier feedback. Experimental results demonstrate that our method substantially enhances both verifiability and functional correctness of code generated by open-source large models: Gemma-4-31B achieves a verification success rate of 90.91%, while GPT-OSS-120B improves from 0% to 81.82% under signature-guided prompting, marking the first systematic validation of open-source models’ potential to produce high-assurance code for complex algorithmic tasks.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large Language Models (LLMs) show promise in automated software engineering, yet their guarantee of correctness is frequently undermined by erroneous or hallucinated code. To enforce model honesty, formal verification requires LLMs to synthesize implementation logic alongside formal specifications that are subsequently proven correct by a mathematical verifier. However, the transition from informal natural language to precise formal specification remains an arduous task. Our work addresses this by providing the NaturalLanguage2VerifiedCode (NL2VC)-60 dataset: a collection of 60 complex algorithmic problems. We evaluate 11 randomly selected problem sets across seven open-weight LLMs using a tiered prompting strategy: contextless prompts, signature prompts providing structural anchors, and self-healing prompts utilizing iterative feedback from the Dafny verifier. To address vacuous verification, where models satisfy verifiers with trivial specifications, we integrate the uDebug platform to ensure functional validation. Our results show that while contextless prompting leads to near-universal failure, structural signatures and iterative self-healing facilitate a dramatic performance turnaround. Specifically, Gemma 4-31B achieved a 90.91\% verification success rate, while GPT-OSS 120B rose from zero to 81.82\% success with signature-guided feedback. These findings indicate that formal verification is now attainable for open-weight LLMs, which serve as effective apprentices for synthesizing complex annotations and facilitating high-assurance software development.

Problem

• Natural Language to Code
• Formal Verification
• LLM Hallucination
• Dafny
• Code Correctness

Innovation

• formal verification
• large language models
• Dafny
• self-healing prompting
• NL2VC dataset