Empirical Characterization of Logging Smells in Machine Learning Code

2026-01-09

arXiv.org

AI Summary

This study addresses the critical gap in systematic research on poor logging practices—referred to as “logging smells”—in machine learning systems, which severely undermine reproducibility, traceability, and observability during model training and deployment. By conducting a large-scale analysis of open-source projects on GitHub, combined with static log pattern analysis and a survey of machine learning engineers, this work systematically identifies and categorizes prevalent logging smells. The paper presents the first taxonomy of logging smells specifically tailored to machine learning contexts, revealing their widespread occurrence, severity, and detrimental impact on system maintainability. This empirical investigation fills a significant void in the literature on logging quality within ML engineering, offering foundational insights for improving software reliability and operational transparency in machine learning pipelines.

Technology Category

• Machine Learning: Scalability of ML Systems
• Knowledge Representation and Reasoning: Logic Programming
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Web Mining and Content Analysis: Web traffic and log analysis
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

\underline{Context:} Logging is a fundamental yet complex practice in software engineering, essential for monitoring, debugging, and auditing software systems. With the increasing integration of machine learning (ML) components into software systems, effective logging has become critical to ensure reproducibility, traceability, and observability throughout model training and deployment. Although various general-purpose and ML-specific logging frameworks exist, little is known about how these tools are actually used in practice or whether ML practitioners adopt consistent and effective logging strategies. To date, no empirical study has systematically characterized recurring bad logging practices--or logging smells--in ML System. \underline{Goal:} This study aims to empirically identify and characterize logging smells in ML systems, providing an evidence-based understanding of how logging is implemented and challenged in practice. \underline{Method:} We propose to conduct a large-scale mining of open-source ML repositories hosted on GitHub to catalogue recurring logging smells. Subsequently, a practitioner survey involving ML engineers will be conducted to assess the perceived relevance, severity, and frequency of the identified smells. \underline{Limitations:} % While The study's limitations include that While our findings may not be generalizable to closed-source industrial projects, we believe our study provides an essential step toward understanding and improving logging practices in ML development.

Problem

• logging smells
• machine learning systems
• empirical study
• software logging
• observability

Innovation

• logging smells
• empirical study
• machine learning systems
• software logging
• observability