When AI Reviews Its Own Code: Recursive Self-Training Collapse in Code LLMs

2026-06-26

AI Summary

This work investigates performance degradation in large language models for code during recursive self-training without external supervision, where training data contamination can lead to collapse. The study systematically examines three gating mechanisms—no review, human gating, and AI self-gating—and reveals for the first time that AI self-gating falls into a “rubber-stamp” regime. It introduces a theoretical framework framing review as distribution reweighting under gating, proving that under self-consistency conditions, AI self-review is equivalent to ungated training. Through empirical evaluation combining compilation checks, static analysis, perplexity, and binary self-assessment with spectral analysis, the experiments demonstrate that ungated training collapses fastest, human gating merely delays degradation, and AI self-gating is initially effective but ultimately fails, underscoring the critical role of external validation in ensuring training stability.

Abstract

Recursive self-training can degrade neural generative models when generated data is reused without fresh human data or external quality control. We study this risk in code LLMs, where AI-generated code can enter real repositories, later become training data, and create a repository-scale self-training loop. While software development traditionally interrupts this loop through pull-request review, tests, compilation, and human approval, AI coding tools now produce code faster than humans can review it, and code review itself is increasingly automated by AI systems. We therefore compare three recursive fine-tuning regimes: no review, Human-gate review using model-independent filters such as compilation and static quality checks, and AI-self-gate review using the code LLM's own signals such as perplexity and binary self-scoring. Across multiple code LLMs and benchmarks, no review collapses fastest, Human-gate filters slow but do not stop collapse, and AI-self-gate filters can look strong early but later lose their filtering effect. In the clearest case, the binary self-gate enters a rubber-stamp regime where acceptance scores rise while benchmark correctness falls. We explain this behavior by formulating review as gated distributional reweighting, proving that AI self-gating degenerates to ungated self-training under a self-confirming acceptance condition, and giving a spectral analysis of representation-level covariance concentration under recursive retraining. These results suggest that stable recursive code LLM training requires exogenous verification rather than model-coupled self-review.

Problem

• recursive self-training
• code LLMs
• self-training collapse
• AI code review
• distributional reweighting

Innovation

• recursive self-training
• code LLMs
• AI self-review
• distributional reweighting
• covariance concentration