Curriculum Reinforcement Learning Can Incentivize Reasoning Capacity in LLMs Beyond the Base Model

2026-06-20

AI Summary

Existing reinforcement learning approaches struggle to transcend the intrinsic reasoning boundaries of large language models, primarily optimizing sampling efficiency within their current capabilities. This work proposes a boundary-aware curriculum reinforcement learning framework that dynamically identifies the model’s current reasoning frontier using pass@k sampling and introduces teacher guidance for samples near or beyond this boundary. By integrating such guidance with reinforcement learning from verifiable rewards (RLVR), the method consolidates newly acquired reasoning patterns. It is the first to combine curriculum learning with explicit boundary-aware mechanisms to actively expand a model’s reasoning capacity. Experiments on Qwen, Llama, and DeepSeek demonstrate consistent improvements in both pass@1 and pass@256 metrics, with an average gain of 9.8 percentage points over the base models and 10.3 percentage points over conventional RLVR.

Abstract

Reinforcement learning with verifiable rewards (RLVR) is widely viewed as a promising path toward continuously improving large language models. Recent works, however, suggest that mainstream RLVR often reallocates sampling probabilities among trajectories already present in the base model: it can improve sampling efficiency, reflected by higher pass@1 scores, but yields limited gains, and can even decrease pass@k scores when k is large, and therefore may fail to expand the base model's reasoning capacity boundary. In this paper, we present a boundary-aware Curriculum RL approach to move beyond the base model's reasoning capacity boundary. Our approach first uses pass@k sampling to locate the current reasoning capacity boundary, then applies targeted teacher guidance to examples near or beyond that boundary, and finally uses RL to consolidate the newly introduced reasoning patterns. Across Qwen, Llama, and DeepSeek base models, boundary-aware Curriculum RL improves both pass@1 scores and pass@256 scores, with pass@1 reflecting one-attempt performance and pass@256 serving as an empirical proxy for the reasoning capacity boundary. In our experiments, average pass@256 improves by 9.8 percentage points over the base models and by 10.3 percentage points over Vanilla RLVR. These results suggest that boundary-aware Curriculum RL can provide a scalable route for LLMs to continuously improve beyond the base model's empirical reasoning capacity boundary.

Problem

• Reinforcement Learning
• Reasoning Capacity
• Large Language Models
• Curriculum Learning
• RLVR

Innovation

• Curriculum Reinforcement Learning
• Reasoning Capacity Boundary
• pass@k Sampling
• Teacher Guidance
• RLVR