WITNESS: A lightweight and practical approach to fine-grained predictive mutation testing

2025-11-14

AI Summary

Existing fine-grained predictive mutation testing relies on deep learning, suffering from high computational overhead and limited support to intra-method mutants only. This paper proposes WITNESS—a lightweight, interpretable, and holistic predictive approach. Methodologically, WITNESS is the first to jointly model both intra- and inter-method mutants, leveraging a novel lightweight feature set that integrates semantic and structural information before and after mutation; it replaces deep models with efficient traditional machine learning to construct a mutant-killing matrix predictor. Evaluated on Defects4J, WITNESS substantially reduces computational cost while achieving state-of-the-art prediction accuracy. Moreover, its test-case prioritization performance closely approximates that of the ground-truth killing matrix and outperforms all baseline methods.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Existing fine-grained predictive mutation testing studies predominantly rely on deep learning, which faces two critical limitations in practice: (1) Exorbitant computational costs. The deep learning models adopted in these studies demand significant computational resources for training and inference acceleration. This introduces high costs and undermines the cost-reduction goal of predictive mutation testing. (2) Constrained applicability. Although modern mutation testing tools generate mutants both inside and outside methods, current fine-grained predictive mutation testing approaches handle only inside-method mutants. As a result, they cannot predict outside-method mutants, limiting their applicability in real-world scenarios. We propose WITNESS, a new fine-grained predictive mutation testing approach. WITNESS adopts a twofold design: (1) With collected features from both inside-method and outside-method mutants, WITNESS is suitable for all generated mutants. (2) Instead of using computationally expensive deep learning, WITNESS employs lightweight classical machine learning models for training and prediction. This makes it more cost-effective and enabling straightforward explanations of the decision-making processes behind the adopted models. Evaluations on Defects4J projects show that WITNESS consistently achieves state-of-the-art predictive performance across different scenarios. Additionally, WITNESS significantly enhances the efficiency of kill matrix prediction. Post-hoc analysis reveals that features incorporating information from before and after the mutation are the most important among those used in WITNESS. Test case prioritization based on the predicted kill matrix shows that WITNESS delivers results much closer to those obtained by using the actual kill matrix, outperforming baseline approaches.

Problem

• Reducing computational costs of predictive mutation testing methods
• Extending applicability to both inside-method and outside-method mutants
• Employing lightweight machine learning for cost-effective mutation prediction

Innovation

• Uses lightweight classical machine learning models
• Handles both inside-method and outside-method mutants
• Incorporates features from before and after mutations