Estimating Correctness Without Oracles in LLM-Based Code Generation

2025-06-26

AI Summary

Evaluating the correctness of code generated by large language models (LLMs) is challenging when no ground-truth oracle (i.e., reference implementation or definitive correctness criterion) is available. Method: This paper proposes an oracle-free evaluation framework based on *incoherence*—a measure quantifying logical contradictions between natural language specifications and generated code. The incoherence score is efficiently computable without code execution or access to correct implementations, and serves as a sound lower bound on error presence. Contribution/Results: The method achieves zero false positives while detecting approximately 67% of erroneous programs. Empirical evaluation shows strong rank correlation (average Spearman ρ > 0.85) between incoherence-based rankings and oracle-based correctness assessments across diverse LLM outputs. By eliminating dependence on oracles, the approach significantly improves evaluation efficiency and scalability, establishing a robust, unsupervised paradigm for code quality assessment.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Generating code from natural language specifications is one of the most successful applications of Large Language Models (LLMs). Yet, they hallucinate: LLMs produce outputs that may be grammatically correct but are factually incorrect. Without an existing, correct implementation (i.e., an oracle), can we quantify how likely the generated program is correct? In this paper, we propose a measure of incorrectness, called incoherence, that can be estimated efficiently in the absence of an oracle and provides a lower bound on the error, i.e., the probability that the LLM-generated program for that specification is incorrect. Our experiments demonstrate an extraordinary effectiveness. For the average code generation task, our incoherence-based methodology can automatically identify about two-thirds of incorrect programs without reports of false positives. In fact, an oracle-based evaluation of LLMs can be reliably replaced by an incoherence-based evaluation. In particular, we find a very strong agreement between the ranking of LLMs by the number of programs deemed correct via an oracle (pass@1) and the ranking of LLMs by the number of programs deemed correct via our incoherence.

Problem

• Estimating correctness of LLM-generated code without oracles
• Quantifying likelihood of factual errors in generated programs
• Automatically identifying incorrect programs without false positives

Innovation

• Estimates program correctness without oracles
• Uses incoherence as incorrectness measure
• Automatically identifies incorrect programs effectively