RV-Syn: Rational and Verifiable Mathematical Reasoning Data Synthesis based on Structured Function Library

2025-04-29

AI Summary

Large language models (LLMs) exhibit limited mathematical reasoning capabilities due to the scarcity of high-quality, logically rigorous, and automatically verifiable training data; conventional synthetic data generation methods fail to ensure internal logical consistency and solution verifiability. Method: We propose an executable computational graph–based, solution-guided problem generation paradigm: a structured mathematical function library is constructed, and executable computational graphs are generated via Python function composition, then reverse-translated into complex, logically sound, and verifiably solvable mathematical problems. Contribution/Results: Our approach is the first to unify endogenous reasoning logic with automatic solution verification, overcoming both the “logical black box” and “verification gap” bottlenecks. Experiments show 100% solution verification accuracy—significantly outperforming existing synthetic methods and human-authored problems—and yield substantial improvements in downstream LLM mathematical reasoning performance.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

The advancement of reasoning capabilities in Large Language Models (LLMs) requires substantial amounts of high-quality reasoning data, particularly in mathematics. Existing data synthesis methods, such as data augmentation from annotated training sets or direct question generation based on relevant knowledge points and documents, have expanded datasets but face challenges in mastering the inner logic of the problem during generation and ensuring the verifiability of the solutions. To address these issues, we propose RV-Syn, a novel Rational and Verifiable mathematical Synthesis approach. RV-Syn constructs a structured mathematical operation function library based on initial seed problems and generates computational graphs as solutions by combining Python-formatted functions from this library. These graphs are then back-translated into complex problems. Based on the constructed computation graph, we achieve solution-guided logic-aware problem generation. Furthermore, the executability of the computational graph ensures the verifiability of the solving process. Experimental results show that RV-Syn surpasses existing synthesis methods, including those involving human-generated problems, achieving greater efficient data scaling. This approach provides a scalable framework for generating high-quality reasoning datasets.

Problem

• Generates verifiable math problems using structured functions
• Ensures logical consistency in problem-solving steps
• Improves dataset quality for LLM reasoning training

Innovation

• Constructs structured mathematical function library
• Generates verifiable computational graphs as solutions
• Back-translates graphs into logic-aware complex problems