Towards Large Reasoning Models: A Survey on Scaling LLM Reasoning Capabilities

2025-01-16

AI Summary

Large language models (LLMs) often struggle with complex problem-solving requiring rigorous analysis, self-reflection, and accurate multi-step reasoning. Method: This work introduces the “Large Reasoning Model” paradigm, proposing a dual-scale scalable reasoning framework: during training, it employs reinforcement learning and automated synthetic reasoning data generation to autonomously construct chain-of-thought (CoT) trajectories; during inference, it integrates autoregressive long-thinking decoding, tree search, and reflection-based modeling to dynamically expand reasoning depth. Contribution/Results: It is the first systematic effort to delineate the core technical pathways for scaling LLM reasoning capabilities—unifying CoT prompting, training-time reasoning learning, and test-time reasoning expansion. Experiments demonstrate substantial improvements in reasoning accuracy across diverse benchmarks. The framework provides both theoretical foundations and practical design principles for next-generation reasoning-optimized architectures, including the o1 series.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Language has long been conceived as an essential tool for human reasoning. The breakthrough of Large Language Models (LLMs) has sparked significant research interest in leveraging these models to tackle complex reasoning tasks. Researchers have moved beyond simple autoregressive token generation by introducing the concept of"thought"-- a sequence of tokens representing intermediate steps in the reasoning process. This innovative paradigm enables LLMs' to mimic complex human reasoning processes, such as tree search and reflective thinking. Recently, an emerging trend of learning to reason has applied reinforcement learning (RL) to train LLMs to master reasoning processes. This approach enables the automatic generation of high-quality reasoning trajectories through trial-and-error search algorithms, significantly expanding LLMs' reasoning capacity by providing substantially more training data. Furthermore, recent studies demonstrate that encouraging LLMs to"think"with more tokens during test-time inference can further significantly boost reasoning accuracy. Therefore, the train-time and test-time scaling combined to show a new research frontier -- a path toward Large Reasoning Model. The introduction of OpenAI's o1 series marks a significant milestone in this research direction. In this survey, we present a comprehensive review of recent progress in LLM reasoning. We begin by introducing the foundational background of LLMs and then explore the key technical components driving the development of large reasoning models, with a focus on automated data construction, learning-to-reason techniques, and test-time scaling. We also analyze popular open-source projects at building large reasoning models, and conclude with open challenges and future research directions.

Problem

• Enhancing Cognitive Abilities
• Large Language Models
• Problem Solving

Innovation

• Reinforcement Learning
• Enhanced Cognitive Abilities
• Optimized Problem Solving Strategies