Fine-Grained Assertion-Based Test Selection

2024-03-24

arXiv.org

AI Summary

To address the high cost of full regression testing in large-scale software and the low precision of existing coarse-grained (class- or method-level) test selection techniques, this paper proposes a fine-grained regression test selection method with **assertions as the minimal selection unit**. Our approach combines static dependency analysis, assertion semantic extraction, and change-impact propagation modeling to precisely identify affected test statements at the statement level. This work is the first to adopt assertions—rather than methods or classes—as the fundamental unit for regression test selection, thereby breaking from conventional paradigms. An empirical evaluation across 11 open-source projects demonstrates that our method reduces average test execution time by 63%, outperforms state-of-the-art techniques by 7–38% in speedup, improves selection precision across all projects, and delivers particularly substantial gains for long-running tests.

Technology Category

• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Dimensionality Reduction/Feature Selection

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

For large software applications, running the whole test suite after each code change is time- and resource-intensive. Regression test selection techniques aim at reducing test execution time by selecting only the tests that are affected by code changes. However, existing techniques select test entities at coarse granularity levels such as test class, which causes imprecise test selection and executing unaffected tests. We propose a novel approach that increases the selection precision by analyzing test code at statement level and treating test assertions as the unit for selection. We implement our fine-grained test selection approach in a tool called oolname and evaluate it by comparing against two state-of-the-art test selection techniques using 11 open-source subjects. Our results show that oolname increases selection precision for all the subjects. Our test selection reduces, on average, 63% of the overall test time, making regression testing 7--38% faster than the other techniques. Our results also indicate that subjects with longer test execution time benefit more by our fine-grained selection technique.

Problem

• Reduces test execution time by selecting affected tests
• Improves selection precision via statement-level assertion analysis
• Accelerates regression testing by minimizing unaffected test runs

Innovation

• Analyzes test code at statement level
• Treats test assertions as selection unit
• Reduces test time by 63% on average