A Systematic Study of Time Limit Exceeded Errors in Online Programming Assignments

2025-10-16

AI Summary

In online programming platforms, Time Limit Exceeded (TLE) errors—characterized by sparse diagnostic feedback and inadequate tooling—frequently lead users to abandon problem-solving after repeated failures. This paper presents the first systematic empirical study of 1,000 real-world TLE submissions, revealing that root causes extend beyond asymptotic algorithmic inefficiency to include infinite loops, misuse of data structures, and I/O bottlenecks. To address this, we propose Nettle, the first automated TLE-specific repair framework. Nettle integrates large language models, compiler-based static analysis, and targeted test generation to perform multi-stage bottleneck localization and synthesize interpretable, functionally correct patches with minimal code changes. Evaluated on the Nettle-Eval benchmark comprising 1,000 real TLE cases, Nettle achieves a 98.5% repair success rate and passes all official platform correctness checks—substantially outperforming prior approaches.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Online programming platforms such as Codeforces and LeetCode attract millions of users seeking to learn to program or refine their skills for industry interviews. A major challenge for these users is the Time Limit Exceeded (TLE) error, triggered when a program exceeds the execution time bound. Although designed as a performance safeguard, TLE errors are difficult to resolve: error messages provide no diagnostic insight, platform support is minimal, and existing debugging tools offer little help. As a result, many users abandon their submissions after repeated TLE failures. This paper presents the first large-scale empirical study of TLE errors in online programming. We manually analyzed 1000 Codeforces submissions with TLE errors, classified their root causes, and traced how users attempted to fix them. Our analysis shows that TLE errors often arise not only from inefficient algorithms but also from infinite loops, improper data structure use, and inefficient I/O, challenging the conventional view that TLEs are purely performance issues. Guided by these findings, we introduce Nettle, the first automated repair tool specifically designed for TLE errors, and Nettle-Eval, the first framework for evaluating TLE repairs. Integrating LLMs with targeted automated feedback generated by the compiler and test cases, Nettle produces small, correct code edits that eliminate TLEs while preserving functionality. Evaluated on the same 1000 real-world cases, Nettle achieves a 98.5% fix rate, far exceeding the strongest LLM baseline, and all of its repairs pass both Nettle-Eval and the platform's official checker, confirming the reliability of our framework.

Problem

• Analyzing root causes of Time Limit Exceeded errors in programming submissions
• Developing automated repair tool for TLE errors using LLMs and compiler feedback
• Evaluating TLE repair framework effectiveness on real-world programming cases

Innovation

• Automated repair tool for TLE errors
• Integrates LLMs with compiler feedback
• Framework evaluates TLE repairs reliably