Learning from Mistakes: Understanding Ad-hoc Logs through Analyzing Accidental Commits

2025-01-17

AI Summary

This study investigates how developers use ad-hoc logging to debug anomalous runtime behavior in asynchronous and callback-heavy JavaScript code. To address the scarcity of empirical evidence, we propose a novel multi-source methodology: mining GitHub Archive for log statements that are hastily committed and subsequently reverted; analyzing live programming videos; performing static pattern matching of logging placements in asynchronous constructs; and conducting expert-informed qualitative synthesis. This yields the first large-scale, manually validated dataset of 548K ad-hoc logs (27 GB) from real-world JavaScript projects. Our empirical analysis reveals—uniquely—that such logs are strongly concentrated along asynchronous execution paths, with pronounced preference for insertion at callback entry/exit points and critical nodes in Promise chains. We release both a reproducible analytical framework and the dataset publicly. This work establishes the first empirically grounded foundation for developing log-driven intelligent debugging tools targeting asynchronous JavaScript.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Search and Optimization: Heuristic Search
• Reasoning under Uncertainty: Probabilistic Programming

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Web traffic and log analysis
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Developers often insert temporary"print"or"log"instructions into their code to help them better understand runtime behavior, usually when the code is not behaving as they expected. Despite the fact that such monitoring instructions, or"ad-hoc logs,"are so commonly used by developers, there is almost no existing literature that studies developers' practices in how they use them. This paucity of knowledge of the use of these ephemeral logs may be largely due to the fact that they typically only exist in the developers' local environments and are removed before they commit their code to their revision control system. In this work, we overcome this challenge by observing that developers occasionally mistakenly forget to remove such instructions before committing, and then they remove them shortly later. Additionally, we further study such developer logging practices by watching and analyzing live-streamed coding videos. Through these empirical approaches, we study where, how, and why developers use ad-hoc logs to better understand their code and its execution. We collect 27 GB of accidental commits that removed 548,880 ad-hoc logs in JavaScript from GitHub Archive repositories to provide the first large-scale dataset and empirical studies on ad-hoc logging practices. Our results reveal several illuminating findings, including a particular propensity for developers to use ad-hoc logs in asynchronous and callback functions. Our findings provide both empirical evidence and a valuable dataset for researchers and tool developers seeking to enhance ad-hoc logging practices, and potentially deepen our understanding of developers' practices towards understanding of software's runtime behaviors.

Problem

• Developer Practices
• Temporary Logs
• Runtime Behavior

Innovation

• Ad-hoc Logging Practices
• Large-scale Dataset
• Runtime Behaviors