DePro: Understanding the Role of LLMs in Debugging Competitive Programming Code

2026-03-19

AI Summary

This work addresses the lack of systematic evaluation of large language models (LLMs) in competitive programming debugging. To this end, we propose DePro, a test-case-driven debugging approach centered on patching rather than rewriting code, which constitutes the first systematic assessment of LLM-based debugging in the Codeforces setting. DePro integrates brute-force reference generation, stress testing, and iterative LLM-based refinement to efficiently localize and correct errors. Evaluated on 13 real-world incorrect submissions, DePro achieves a 100% repair success rate, reduces the number of debugging attempts by up to 64%, and saves an average of 7.6 minutes per problem, substantially improving debugging efficiency.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Reasoning under Uncertainty: Probabilistic Programming

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Debugging consumes a substantial portion of the software development lifecycle, yet the effectiveness of Large Language Models(LLMs) in this task is not well understood. Competitive programming offers a rich benchmark for such evaluation, given its diverse problem domains and strict efficiency requirements. We present an empirical study of LLM-based debugging on competitive programming problems and introduce DePro, a test-case driven approach that assists programmers by correcting existing code rather than generating new solutions. DePro combines brute-force reference generation, stress testing, and iterative LLM-guided refinement to identify and resolve errors efficiently.Experiments on 13 faulty user submissions from Codeforces demonstrate that DePro consistently produces correct solutions, reducing debugging attempts by up to 64% and debugging time by an average of 7.6 minutes per problem compared to human programmers and zero-shot LLM debugging.

Problem

• debugging
• Large Language Models
• competitive programming
• code correction
• software development

Innovation

• LLM-based debugging
• test-case driven
• iterative refinement
• competitive programming
• code repair