VeriEvol: Scaling Multimodal Mathematical Reasoning via Verifiable Evol-Instruct

2026-06-22

AI Summary

This work proposes VeriEvol, a verifiable evolution framework that decouples prompt difficulty from answer reliability to enhance data quality and challenge for reinforcement learning in visual mathematical reasoning. The approach employs a type-aware evolution module to generate challenging multimodal prompts and introduces a hypothesis-testing verifier agent (HTV-Agent) that leverages multi-source refutation mechanisms to ensure answer correctness. This pipeline produces high-quality, evolved supervised fine-tuning (SFT) data tailored for GRPO-style reinforcement learning. Evaluated across five visual math benchmarks, VeriEvol scales the SFT dataset from 10K to 250K samples, improving average accuracy from 35.42% to 54.73% and yielding a +3.88% performance gain over non-evolved RL baselines. The complete verification trajectories are open-sourced to facilitate auditable research.

Abstract

Scaling reinforcement learning for visual mathematical reasoning requires more than generating harder questions: as data volume grows, the reward labels themselves must remain reliable. Yet existing data pipelines scale supervision while trusting the labeller, and policy-side methods assume the underlying answers are already correct. We instead treat scaling as a verifiable data-construction problem and decouple two axes before any policy update: prompt difficulty, expanded by route-specific evolution operators, and answer reliability, enforced by offline hypothesis-test falsification. We instantiate this as VeriEvol, an iterative framework with two extensible components: a type-aware evolution module that rewrites low-difficulty image-question seeds into harder, image-grounded prompts; and HTV-Agent, a verifier that accepts an answer only after multi-source counter-evidence has failed to refute it. The resulting verified data scales in volume, extends by adding evolution routes or verifier channels, and plugs directly into existing GRPO-style RL recipes. On a five-benchmark visual-math suite, scaling evolved SFT data from 10K to 250K samples raises the mean accuracy from 35.42 to 54.73; then, with backbone, SFT initialization, and GRPO recipe held fixed, VeriEvol adds a cumulative +3.88 over an un-evolved RL baseline, of which +1.82 comes from evolved prompts and +2.06 from the HTV-Agent verifier. We release the prompts, data, models, code, and the full verifier trace of every sample, so that downstream work can scale and audit the pipeline rather than only inspect its outputs.

Problem

• visual mathematical reasoning
• reliable supervision
• reward labeling
• data scaling
• answer verification

Innovation

• Verifiable Evol-Instruct
• Multimodal Mathematical Reasoning
• Data Construction
• Hypothesis-Test Falsification
• Reinforcement Learning Scaling