Diverse Inference and Verification for Advanced Reasoning

2025-02-14

AI Summary

This work addresses the limited performance of state-of-the-art reasoning large language models (e.g., o1, o3, R1) on high-level abstract reasoning benchmarks—including IMO combinatorics problems, ARC puzzles, and Humanity’s Last Exam (HLE). We propose a test-time diversified reasoning framework leveraging multi-model collaboration. Our contributions are threefold: (1) a cross-task adaptive verification mechanism integrating Lean theorem proving, programmatic ARC solvers, and automated code verification; (2) embedding test-time Monte Carlo simulation, reinforcement learning, and meta-learning into reasoning graph optimization to enable dynamic prompt, strategy, and data adaptation; and (3) synergistic integration of Best-of-N rejection sampling with multi-model ensembling. Experiments demonstrate substantial gains: IMO combinatorics accuracy improves from 33.3% to 77.8%, HLE accuracy rises from 8% to 37%, and 80% of the 948 previously unsolved ARC puzzles are successfully resolved—surpassing o3 by 26.5% in overall performance.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Search and Optimization: Metareasoning and Metaheuristics
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Reasoning LLMs such as OpenAI o1, o3 and DeepSeek R1 have made significant progress in mathematics and coding, yet find challenging advanced tasks such as International Mathematical Olympiad (IMO) combinatorics problems, Abstraction and Reasoning Corpus (ARC) puzzles, and Humanity's Last Exam (HLE) questions. We use a diverse inference approach that combines multiple models and methods at test time. We find that verifying mathematics and code problems, and rejection sampling on other problems is simple and effective. We automatically verify correctness of solutions to IMO problems by Lean, and ARC puzzles by code, and find that best-of-N effectively answers HLE questions. Our approach increases answer accuracy on IMO combinatorics problems from 33.3% to 77.8%, accuracy on HLE questions from 8% to 37%, and solves 80% of ARC puzzles that 948 humans could not and 26.5% of ARC puzzles that o3 high compute does not. Test-time simulations, reinforcement learning, and meta-learning with inference feedback improve generalization by adapting agent graph representations and varying prompts, code, and datasets. Our approach is reliable, robust, and scalable, and in the spirit of reproducible research, we will make it publicly available upon publication.

Problem

• Improves accuracy on complex reasoning tasks
• Verifies solutions using diverse inference methods
• Enhances generalization with adaptive learning techniques

Innovation

• Combines multiple models and methods
• Automatically verifies solutions using Lean and code
• Improves generalization with reinforcement learning