AutoPyVerifier: Learning Compact Executable Verifiers for Large Language Model Outputs

2026-04-24

AI Summary

Existing approaches to verifying outputs of large language models (LLMs) struggle to balance expressiveness and reliability: LLM-based verifiers are flexible yet unreliable, while deterministic verifiers are reliable but limited in expressive power. This work proposes AutoPyVerifier, a framework that leverages LLMs to generate candidate Python verification functions and employs a directed acyclic graph (DAG)-based search mechanism to systematically explore the space of executable verifiers, automatically inducing a minimal set that approximates target judgment criteria—such as correctness. AutoPyVerifier is the first method to enable data-driven, automatic synthesis of compact, executable, and semantically structured verifiers. Evaluated on tasks including mathematical reasoning and code generation, it achieves up to a 55.0-point F1 improvement; when integrated back into LLMs as an external tool, it boosts downstream task accuracy by up to 17.0 percentage points.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Verification is becoming central to both reinforcement-learning-based training and inference-time control of large language models (LLMs). Yet current verifiers face a fundamental trade-off: LLM-based verifiers are expressive but hard to control and prone to error, while deterministic executable verifiers are reliable and interpretable but often limited in capability. We study the following question: given a development set of LLM outputs and labels for a target objective, such as correctness, can we automatically induce a minimal set of Python verifiers whose joint satisfaction closely matches that objective? We propose AutoPyVerifier, a framework that uses an LLM to synthesize candidate verifier functions and then refines them through search over a directed acyclic graph (DAG). By navigating the DAG, AutoPyVerifier systematically explores the space of deterministic executable verifiers and selects a compact verifier set whose joint satisfaction best approximates the target objective. Across mathematical reasoning, coding, function calling, and instruction-following benchmarks for several state-of-the-art LLMs, AutoPyVerifier improves target-objective prediction by up to 55.0 F1 points over the initial LLM-generated verifier sets. Additional analyses show that the most useful verification targets vary by benchmark and model, and that the DAG-based search shifts the learned verifier sets toward more structural and semantically grounded checks. We further show that exposing the discovered verifier set to an LLM as an external tool improves downstream accuracy by up to 17.0 points. We release our code

Problem

• verifier
• large language models
• executable verification
• correctness
• automated synthesis

Innovation

• executable verifiers
• LLM output verification
• program synthesis
• DAG-based search
• deterministic verification