On Accelerating Deep Neural Network Mutation Analysis by Neuron and Mutant Clustering

2025-01-22

AI Summary

To address the high computational cost and poor scalability of mutation analysis for deep neural networks (DNNs), this paper proposes DEEPMAACC—a novel framework that introduces hierarchical agglomerative clustering (HAC) jointly at both the neuron and mutant levels. First, neurons are clustered based on weight similarity to reduce the number of mutants generated; second, the resulting mutants are further clustered, and representative samples are selected for testing. Under a controlled error bound of ±3% in mutation scores, DEEPMAACC achieves dual-speedup: neuron-level clustering yields an average 69.8% acceleration (with a −26.8% error in mutation score estimation), while mutant-level clustering achieves 35.3% speedup (with a +2.0% error). Extensive experiments across eight DNN models, four benchmark datasets, and multiple classification-oriented mutation operators—including weight perturbation and layer deletion—demonstrate its effective trade-off between efficiency and accuracy.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Search and Optimization: Evolutionary Computation
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements

Abstract

Mutation analysis of deep neural networks (DNNs) is a promising method for effective evaluation of test data quality and model robustness, but it can be computationally expensive, especially for large models. To alleviate this, we present DEEPMAACC, a technique and a tool that speeds up DNN mutation analysis through neuron and mutant clustering. DEEPMAACC implements two methods: (1) neuron clustering to reduce the number of generated mutants and (2) mutant clustering to reduce the number of mutants to be tested by selecting representative mutants for testing. Both use hierarchical agglomerative clustering to group neurons and mutants with similar weights, with the goal of improving efficiency while maintaining mutation score. DEEPMAACC has been evaluated on 8 DNN models across 4 popular classification datasets and two DNN architectures. When compared to exhaustive, or vanilla, mutation analysis, the results provide empirical evidence that neuron clustering approach, on average, accelerates mutation analysis by 69.77%, with an average -26.84% error in mutation score. Meanwhile, mutant clustering approach, on average, accelerates mutation analysis by 35.31%, with an average 1.96% error in mutation score. Our results demonstrate that a trade-off can be made between mutation testing speed and mutation score error.

Problem

• Deep Neural Networks
• Mutation Analysis
• Computational Efficiency

Innovation

• DEEPMAACC
• accelerated mutation analysis
• deep learning robustness