ReflexiCoder: Teaching Large Language Models to Self-Reflect on Generated Code and Self-Correct It via Reinforcement Learning

2026-03-06

AI Summary

This work proposes ReflexiCoder-8B, a novel approach that internalizes the complete reasoning trajectory—encompassing code generation, self-reflection, and self-correction—directly into the model weights via reinforcement learning, eliminating the need for external execution engines or ground-truth labels. Addressing the limited single-pass code generation performance of large language models on complex algorithmic tasks and the high computational cost or reliance on external feedback in existing iterative refinement methods, ReflexiCoder-8B leverages an RL-zero training paradigm with a fine-grained reward function to significantly enhance both reasoning efficiency and accuracy. The model achieves state-of-the-art results among open-source models across multiple benchmarks, attaining a HumanEval score of 94.51% while reducing inference-time computational overhead by approximately 40%.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

While Large Language Models (LLMs) have revolutionized code generation, standard"System 1"approaches, generating solutions in a single forward pass, often hit a performance ceiling when faced with complex algorithmic tasks. Existing iterative refinement strategies attempt to bridge this gap at inference time, yet they predominantly rely on external oracles, execution feedback, or computationally expensive prompt-response cycles. In this work, we propose ReflexiCoder, a novel reinforcement learning (RL) framework that internalizes the structured reasoning trajectory, encompassing initial generation, bug and optimization aware reflection, and self-correction, directly into the model's weights. Unlike prior methods, ReflexiCoder shifts the paradigm from external-dependent refinement to an intrinsic, fully autonomous self-reflection and self-correction capabilities at inference time. We utilize an RL-zero training paradigm with granular reward functions to optimize the entire reflection-correction trajectory, teaching the model how to debug without reliance on ground-truth feedback or execution engines at inference time. Extensive experiments across seven benchmarks demonstrate that our ReflexiCoder-8B establishes a new state-of-the-art (SOTA) among leading open-source models in the 1.5B-14B range, achieving 94.51% (87.20%) on HumanEval (Plus), 81.80% (78.57%) on MBPP (Plus), 35.00% on BigCodeBench, 52.21% on LiveCodeBench, and 37.34% on CodeForces in a single-attempt setting, rivaling or surpassing proprietary models like GPT-5.1. Notably, our framework is significantly more token-efficient than base models, reducing inference-time compute overhead by approximately 40% through disciplined, high-speed reasoning and reflection patterns. Source code is available at https://github.com/juyongjiang/ReflexiCoder.

Problem

• code generation
• self-reflection
• self-correction
• large language models
• algorithmic reasoning

Innovation

• self-reflection
• self-correction
• reinforcement learning
• code generation
• LLM fine-tuning