From Code Smells to Best Practices: Tackling Resource Leaks in PyTorch, TensorFlow, and Keras

2025-11-19

AI Summary

Contemporary machine learning research overemphasizes model performance while neglecting resource efficiency and long-term sustainability. Method: This paper systematically identifies memory/GPU-memory leak–inducing code smells in PyTorch, TensorFlow, and Keras—derived from developer community discussions and real-world code snippets—yielding 30 PyTorch-specific and 16 TensorFlow/Keras-specific anti-patterns. It proposes the first cross-framework resource leakage taxonomy, balancing generality with framework-specific adaptability, validated through a three-stage empirical process: qualitative analysis, normative refinement, and practical evaluation. Contribution/Results: The study yields 50 actionable best practices for mitigating resource leaks. It fills a critical gap in ML engineering research on resource efficiency and provides a structured, methodology-grounded coding guide for building efficient, robust, and sustainable machine learning systems.

Technology Category

• Machine Learning: Hardware-aware ML
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Security and Privacy: Security and privacy of machine learning and AI applications
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Much of the existing ML research focuses on model performance metrics, leaving limited attention to the long-term sustainability and resource efficiency of ML applications. While high performance is essential, ensuring efficient resource management is equally critical for robust deployment. This study addresses this gap by systematically identifying code smells that lead to resource leaks in ML applications. We conducted an empirical investigation of developer discussions and real-world code snippets from PyTorch, TensorFlow, and Keras. The analysis identified 30 PyTorch-related smells and 16 TensorFlow/Keras smells linked to resource leaks. These smells were categorized in two ways: (1) based on their root causes, and (2) as general ML smells with framework-specific characteristics. For each smell, we derived at least one best practice, resulting in 50 recommended coding patterns aimed at reducing resource leakage and improving efficiency. To ensure the validity of our findings, we employed a three-phase validation process involving independent analysis by three authors followed by consensus discussions. This is the first comprehensive study to examine resource-leak-inducing code smells across major ML frameworks and to present actionable best practices for mitigating them. The contributions support developers in building more efficient and sustainable ML applications and offer a structured view of the underlying causes of resource leaks.

Problem

• Identifying code smells causing resource leaks in PyTorch, TensorFlow and Keras
• Categorizing 46 framework-specific resource leak patterns by root causes
• Providing 50 actionable best practices to improve ML application efficiency

Innovation

• Systematically identified code smells causing resource leaks
• Categorized smells by root causes and framework-specific characteristics
• Derived 50 best practices to reduce resource leakage