TD-Suite: All Batteries Included Framework for Technical Debt Classification

2025-04-15

AI Summary

Technical debt classification lacks automated support, hindering precise governance. This paper proposes the first end-to-end, fine-grained technical debt classification framework: it leverages Transformer-based models to parse textual artifacts (e.g., issue reports) for binary detection and multi-class categorization (e.g., code, design, documentation debt). To address severe class imbalance and reduce environmental impact, we innovatively integrate carbon-aware training—incorporating class-weighted loss, early stopping, and k-fold cross-validation—thereby improving robustness while lowering AI training carbon footprint. The framework is delivered as a Docker-packaged Gradio web interface, enabling immediate deployment and interactive analysis. Evaluated on diverse real-world datasets, our approach achieves high classification accuracy (F1 > 0.89), significantly enhancing the precision, interpretability, and engineering practicality of technical debt identification.

Technology Category

• Machine Learning: Hardware-aware ML
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Computer Vision: Object Detection & Categorization

Application Category

• Responsible Web: Sustainability and climate impact of web technologies
• Systems and Infrastructure for Web, Mobile and WoT: Sustainability and carbon-aware systems for Web, mobile, and WoT
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Recognizing that technical debt is a persistent and significant challenge requiring sophisticated management tools, TD-Suite offers a comprehensive software framework specifically engineered to automate the complex task of its classification within software projects. It leverages the advanced natural language understanding of state-of-the-art transformer models to analyze textual artifacts, such as developer discussions in issue reports, where subtle indicators of debt often lie hidden. TD-Suite provides a seamless end-to-end pipeline, managing everything from initial data ingestion and rigorous preprocessing to model training, thorough evaluation, and final inference. This allows it to support both straightforward binary classification (debt or no debt) and more valuable, identifying specific categories like code, design, or documentation debt, thus enabling more targeted management strategies. To ensure the generated models are robust and perform reliably on real-world, often imbalanced, datasets, TD-Suite incorporates critical training methodologies: k-fold cross-validation assesses generalization capability, early stopping mechanisms prevent overfitting to the training data, and class weighting strategies effectively address skewed data distributions. Beyond core functionality, and acknowledging the growing importance of sustainability, the framework integrates tracking and reporting of carbon emissions associated with the computationally intensive model training process. It also features a user-friendly Gradio web interface in a Docker container setup, simplifying model interaction, evaluation, and inference.

Problem

• Automates technical debt classification in software projects
• Identifies specific debt types like code or design issues
• Ensures robust model performance on imbalanced datasets

Innovation

• Uses transformer models for text analysis
• Provides end-to-end debt classification pipeline
• Incorporates robust training and carbon tracking