Leveraging Stack Traces for Spectrum-based Fault Localization in the Absence of Failing Tests

2024-05-01

arXiv.org

AI Summary

Spectrum-Based Fault Localization (SBFL) fails when no failing tests are available to trigger faults. Method: This paper systematically demonstrates, for the first time, that stack traces from crash reports can serve as pseudo-failure signals in lieu of actual failing tests, and proposes SBEST—a novel SBFL method that integrates exception-location semantics with method-call-graph reachability to embed stack-trace information into the spectrum analysis framework. SBEST jointly leverages test coverage matrices and parsed stack traces to enable precise fault localization even in the absence of failing tests. Results: Experiments show SBEST improves Mean Average Precision (MAP) by 32.22% and Mean Reciprocal Rank (MRR) by 17.43% over the baseline MAP method. Moreover, 98.3% of defect-fixing intentions align with stack-trace anomalies, and 78.3% of defective methods are reachable within an average of 0.34 call-graph hops. This work establishes a new lightweight, crash-driven paradigm for fault localization.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Search and Optimization: Local Search
• Machine Learning: Evaluation and Analysis

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Bug fixing is a crucial task in software maintenance to hold user trust. Although various automated fault localization techniques exist, they often require specific conditions to be effective. For example, Spectrum-Based Fault Localization (SBFL) techniques need at least one failing test to identify bugs, which may not always be available. Bug reports, particularly those with stack traces, provide detailed information on system execution failures and are invaluable for developers. This study focuses on utilizing stack traces from crash reports as fault-triggering tests for SBFL. Our findings indicate that only 3.33% of bugs have fault-triggering tests, limiting traditional SBFL efficiency. However, 98.3% of bugfix intentions align directly with exceptions in stack traces, and 78.3% of buggy methods are reachable within an average of 0.34 method calls, proving stack traces as a reliable source for locating bugs. We introduce a new approach, SBEST, that integrates stack trace data with test coverage to enhance fault localization. Our approach shows a significant improvement, increasing Mean Average Precision (MAP) by 32.22% and Mean Reciprocal Rank (MRR) by 17.43% over traditional stack trace ranking methods.

Problem

• Proposes SBEST for fault localization without fault-trigging tests
• Uses stack traces as proxies for missing fault-triggering tests
• Improves localization accuracy when crash reports lack test cases

Innovation

• Uses stack traces as fault-triggering test proxies
• Integrates stack trace with test coverage data
• Improves fault localization without failing tests