AutoICE: Automatically Synthesizing Verifiable C Code via LLM-driven Evolution

2025-12-08

AI Summary

Generating verifiable C code from natural language requirements remains challenging, as existing approaches suffer from high rates of syntactic and semantic errors—yielding low verification pass rates (~65%) due to insufficient domain-specific corpora and inadequate modeling of implicit knowledge. This paper proposes an LLM-driven evolutionary search framework. It introduces diverse individual initialization and cooperative crossover to mitigate error accumulation in single-agent iterative refinement, and incorporates a self-reflective mutation strategy to explicitly uncover and formalize implicit requirement knowledge. By tightly integrating large language models, evolutionary algorithms, and formal verification feedback, the framework jointly optimizes for syntactic correctness, semantic fidelity, and verification compliance. Evaluated on standard benchmarks, our method achieves a verification pass rate of 90.36%; on a developer-friendly dataset, it attains 88.33%—significantly outperforming state-of-the-art methods.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Search and Optimization: Evolutionary Computation
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Automatically synthesizing verifiable code from natural language requirements ensures software correctness and reliability while significantly lowering the barrier to adopting the techniques of formal methods. With the rise of large language models (LLMs), long-standing efforts at autoformalization have gained new momentum. However, existing approaches suffer from severe syntactic and semantic errors due to the scarcity of domain-specific pre-training corpora and often fail to formalize implicit knowledge effectively. In this paper, we propose AutoICE, an LLM-driven evolutionary search for synthesizing verifiable C code. It introduces the diverse individual initialization and the collaborative crossover to enable diverse iterative updates, thereby mitigating error propagation inherent in single-agent iterations. Besides, it employs the self-reflective mutation to facilitate the discovery of implicit knowledge. Evaluation results demonstrate the effectiveness of AutoICE: it successfully verifies $90.36$% of code, outperforming the state-of-the-art (SOTA) approach. Besides, on a developer-friendly dataset variant, AutoICE achieves a $88.33$% verification success rate, significantly surpassing the $65$% success rate of the SOTA approach.

Problem

• Automatically synthesizes verifiable C code from natural language requirements
• Addresses syntactic and semantic errors in existing autoformalization approaches
• Enhances discovery of implicit knowledge to improve verification success rates

Innovation

• LLM-driven evolutionary search synthesizes verifiable C code
• Diverse initialization and collaborative crossover enable iterative updates
• Self-reflective mutation facilitates discovery of implicit knowledge