Improving LLM Code Generation via Requirement-Aware Curriculum Reinforcement Learning

2026-05-01

AI Summary

This work addresses the limited code generation capabilities of current large language models on complex programming tasks and the inadequacy of conventional curriculum reinforcement learning methods in accurately perceiving and optimizing requirement difficulty, compounded by suboptimal sampling strategies. To overcome these challenges, the authors propose RECRL, a novel framework that integrates software requirements engineering principles into curriculum reinforcement learning for the first time. RECRL automatically evaluates requirement difficulty, prioritizes high-difficulty requirements for optimization, and employs an adaptive curriculum sampling mechanism to construct training batches with smoothly increasing complexity. Extensive experiments across five mainstream code generation benchmarks demonstrate that RECRL consistently outperforms five state-of-the-art baselines, achieving average Pass@1 improvements of 1.23%–5.62% and significantly enhancing both training efficiency and code generation quality.

Abstract

Code generation, which aims to automatically generate source code from given programming requirements, has the potential to substantially improve software development efficiency. With the rapid advancement of large language models (LLMs), LLM-based code generation has attracted widespread attention from both academia and industry. However, as programming requirements become increasingly complex, existing LLMs still exhibit notable performance limitations. To address this challenge, recent studies have proposed training-based curriculum reinforcement learning (CRL) strategies to improve LLM code generation performance. Despite their effectiveness, existing CRL approaches suffer from several limitations, including misaligned requirement difficulty perception, the absence of requirement difficulty optimization, and suboptimal curriculum sampling strategies. In CRL-based code generation, programming requirements serve as the sole input to the model, making their quality and difficulty critical to training effectiveness. Motivated by insights from software requirements engineering, we propose RECRL, a novel requirement-aware curriculum reinforcement learning framework for enhancing LLM-based code generation. RECRL automatically perceives model-specific requirement difficulty, optimizes challenging requirements to improve training data utilization, and employs an adaptive curriculum sampling strategy to construct training batches with smoothly varying difficulty. Extensive experiments on five state-of-the-art LLMs across five widely-used code generation benchmarks by comparing with five state-of-the-art baselines, demonstrate the significant effectiveness of RECRL. For example, RECRL achieves an average Pass@1 improvement of 1.23%-5.62% over all state-of-the-art baselines.

Problem

• code generation
• large language models
• curriculum reinforcement learning
• requirement difficulty
• software development

Innovation

• requirement-aware
• curriculum reinforcement learning
• code generation
• difficulty perception
• adaptive sampling