How Powerful are LLMs in Generating Formal Program Specifications?

2026-08-13

AI Summary

This work addresses the challenge of accurately evaluating the quality of formal program specifications generated by large language models (LLMs), a task often confounded by the inherent complexity of formal verification. To mitigate this issue, the authors propose Coins, a framework built upon the Rocq formal system that aligns evaluation with the asymmetric nature of formal reasoning. Coins instantiates generated specifications against trusted test cases and automatically derives concrete proof obligations, using test-case-driven proof success rates as a reliable proxy for specification quality—thereby avoiding misattribution of proof difficulty to specification inadequacy. Large-scale experiments on HumanEval demonstrate that current LLMs still face substantial difficulties in specification generation, and that Coins provides a more accurate assessment of genuine differences in specification quality.

Abstract

Formal verification provides strong guarantees of software correctness, but its adoption is limited by the high cost of writing precise formal specifications. While recent large language models (LLMs) have shown strong capabilities in theorem proving and verified code generation, their true ability to generate program specifications remains unclear. Existing evaluations require either verifying implementation conformance or proving semantic equivalence between specifications, both of which are formidably difficult and may conflate proof difficulty with specification quality. To address this problem, we introduce Coins, a Rocq based evaluation framework that assesses specification quality by instantiating specifications under evaluation on trusted test cases and generating concrete proof obligations. This design aligns with the asymmetric nature of formal reasoning, where successful proofs provide reliable evidence while proof failures are inherently ambiguous. Using Coins, we conduct a large scale study on HumanEval with a curated set of human written Rocq specifications. Our results show that specification generation remains a formidable challenge, and that verification complexity can obscure genuine differences in specification quality. Overall, we find that accurate specification evaluation, rather than model scaling alone, is central to understanding the power of LLMs for specification synthesis, and that test case based formal reasoning offers a more faithful and discriminative measure of progress.

Problem

• formal specification
• large language models
• specification evaluation
• formal verification
• program synthesis

Innovation

• formal specification
• large language models
• evaluation framework
• test-case-based reasoning
• proof obligations