Learning from Contrasts: Synthesizing Reasoning Paths from Diverse Search Trajectories

2026-04-13

AI Summary

Traditional Monte Carlo Tree Search (MCTS) in supervised signal extraction retains only the highest-reward trajectory, discarding rich contrastive information embedded in numerous explored paths and thereby suffering from low data efficiency. This work proposes the Contrastive Reasoning Path Synthesis (CRPS) framework, which explicitly models the distinctions between high- and low-quality reasoning paths through multi-trajectory contrastive analysis and a structured reflection mechanism. CRPS synthesizes novel reasoning chains that integrate successful patterns while avoiding failure pitfalls, shifting supervised signal generation from selection to synthesis. This paradigm substantially enhances data utilization: a model trained on merely 60,000 CRPS-generated samples surpasses one trained on 590,000 conventional samples—a nearly tenfold improvement in data efficiency—and demonstrates stronger out-of-domain generalization on benchmark evaluations.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Monte Carlo Tree Search (MCTS) has been widely used for automated reasoning data exploration, but current supervision extraction methods remain inefficient. Standard approaches retain only the single highest-reward trajectory, discarding the comparative signals present in the many explored paths. Here we introduce \textbf{Contrastive Reasoning Path Synthesis (CRPS)}, a framework that transforms supervision extraction from a filtering process into a synthesis procedure. CRPS uses a structured reflective process to analyze the differences between high- and low-quality search trajectories, extracting explicit information about strategic pivots and local failure modes. These insights guide the synthesis of reasoning chains that incorporate success patterns while avoiding identified pitfalls. We show empirically that models fine-tuned on just 60K CRPS-synthesized examples match or exceed the performance of baselines trained on 590K examples derived from standard rejection sampling, a 20$\times$ reduction in dataset size. Furthermore, CRPS improves generalization on out-of-domain benchmarks, demonstrating that learning from the contrast between success and failure produces more transferable reasoning capabilities than learning from success alone.

Problem

• Monte Carlo Tree Search
• supervision extraction
• reasoning paths
• contrastive learning
• automated reasoning

Innovation

• Contrastive Reasoning Path Synthesis
• Monte Carlo Tree Search
• reasoning path synthesis
• failure-aware learning
• data-efficient fine-tuning