software testing

This study systematically investigates how artificial intelligence (AI) hierarchically augments software testing—from zero to full automation—and identifies emergent AI-driven testing paradigms. Method: We propose AI4ST, the first ontology-based classification framework for AI-enhanced software testing, constructed via ontological engineering and integrating systematic literature review (SLR) with domain knowledge modeling. Contribution/Results: AI4ST provides a unified, structured taxonomy characterizing recent AI applications across test generation, execution, analysis, and maintenance. It precisely pinpoints critical technical gaps—including explainability, generalization capability, and test trustworthiness assurance—as well as interdisciplinary research opportunities. The framework establishes an extensible theoretical foundation and empirical analysis toolkit, thereby advancing the standardization and deep evolution of AI-driven software testing.

This study addresses the imbalance in the test pyramid—characterized by an overreliance on coarse-grained integration and system tests, which leads to difficulties in fault localization and slow execution—by proposing, for the first time, a method to automatically generate unit tests from existing integration tests. The approach combines static and dynamic analysis to automatically isolate component dependencies and enhance coverage at the unit level. Implemented as a Node.js tool and evaluated on twelve open-source JavaScript projects, the technique produces high-quality unit tests that significantly improve test suite structure, thereby increasing both testing efficiency and maintainability.

Existing code-level formal verification tools scale poorly to large-scale software, while mainstream unit-level verification relies heavily on manual effort, often missing critical defects. This paper proposes the “Unit Proof Framework” research agenda—the first systematic definition of a unit verification paradigm supporting automated decoupling and independent verification of code units. Methodologically, it integrates formal verification, program analysis, modular verification, and automated toolchain design, with deep alignment to industrial development practices (e.g., AWS workflows). Its core contributions include: (1) establishing a scalable, engineering-friendly unit verification methodology; (2) characterizing a taxonomy of key technical challenges; (3) overcoming bottlenecks inherent in manual verification; and (4) significantly improving early detection of code-level defects. Collectively, this work lays the theoretical foundation and provides a practical technical pathway for building high-assurance, deployable automated verification infrastructure.

To address the insufficient speed, reliability, and maintainability of testing in modern software systems, this paper designs and implements a modular automated testing framework that deeply integrates Cucumber-BDD with Java. The framework introduces a novel natural-language-driven test design and engineering implementation co-development mechanism, supporting dynamic environment adaptation, reusable component-based architecture, and end-to-end automated reporting with closed-loop feedback. It integrates Selenium, TestNG, Maven, and Jenkins to enable seamless embedding into CI/CD pipelines. Empirical evaluation demonstrates a reduction of manual testing effort by over 40%, a 35% improvement in defect detection rate, and a 50% decrease in script maintenance cost. These outcomes significantly enhance agility in iterative development and streamline multi-environment one-click deployment efficiency.

Modern distributed software QA faces escalating quality costs due to high complexity, rapid iteration, and constrained resources. This paper conducts a systematic empirical evaluation of AI-based tools—including large language models (LLMs), AI agents, mutation testing, and static analysis—across exploratory testing, equivalence class analysis, test generation, test suite optimization, and end-to-end regression testing. It presents the first feasibility validation of AI agent–driven end-to-end regression testing and uncovers fundamental limitations of LLMs in semantic equivalence coverage, interpretability, and bias mitigation, proposing a complementary verification methodology. A proof-of-concept implementation demonstrates an 8.3% reduction in test execution volatility, yielding significant gains in efficiency and coverage breadth. Concurrently, the study identifies three critical deployment bottlenecks: poor interpretability, opaque decision-making (black-box behavior), and inconsistent semantic coverage.

High development and maintenance costs of test automation scripts, coupled with strong programming expertise requirements, hinder widespread adoption in software testing. Method: Integrating analysis of 342 gray-literature sources with semi-structured interviews involving five domain experts, this study constructs the first dual-dimensional taxonomy categorizing both TA challenges and corresponding AI-driven solutions. Based on this framework, we systematically analyze 100 AI-powered testing tools to build an evidence-based tool atlas and establish precise “problem–solution–tool” mappings. Contribution/Results: We identify automated test generation and self-healing scripts as the two dominant AI paradigms in TA; further, we pinpoint five high-adoption benchmark tools—including Applitools—through empirical evaluation. The resulting taxonomy, atlas, and mappings provide industry practitioners with evidence-based guidance for tool selection and offer researchers concrete directions for advancing AI-augmented quality assurance.

This study addresses the lack of explicit justification in AI-generated test cases, which hinders engineers’ ability to understand and evaluate their rationale. To bridge this gap, the work introduces argumentation structures into AI-based test generation for the first time, proposing a conceptual taxonomy and a structured template grounded in four core elements: test objectives, claims, reasons, and evidence. By integrating conceptual modeling with argumentation theory, the approach establishes an interpretable test generation framework that standardizes the articulation of justifications. This mechanism substantially enhances the understandability, traceability, and trustworthiness of AI-generated test cases, enabling more effective human review and validation in software testing processes.

This work proposes a novel paradigm that bridges the long-standing divide between testing and formal verification in traditional software validation, enabling them to synergistically enhance both efficiency and quality. Grounded in Design by Contract, the approach leverages the counterexample generation capability of SMT solvers to transform formal verification tools into an integrated engine for automated testing and repair. Within a unified framework, the method simultaneously achieves three key objectives: automatic generation of test cases for faulty programs, construction of regression test suites with full coverage for correct programs, and correctness-guaranteed program repair. This represents the first integration of verification, testing, and repair into a single cohesive methodology.

This work addresses the challenge that rapid software development often compromises code maintainability, thereby hindering safe AI-assisted refactoring. To mitigate this, the authors propose an iterative refactoring approach that integrates large language models with human oversight. The method first leverages a code-specialized large language model to automatically generate high-coverage unit tests that capture existing program behavior. Subsequently, developers guide test-driven refactoring, while branch coverage metrics are used to constrain and validate model-generated outputs. Empirical evaluation demonstrates that the approach produces nearly 16,000 lines of reliable test code within hours, achieving up to 78% branch coverage on critical modules. This significantly reduces regression risk during large-scale refactoring and enhances the reliability and practicality of AI-assisted code restructuring.

Existing AI-based test generation approaches typically produce static, one-off outputs that often yield invalid, redundant, or non-executable test cases and lack mechanisms for execution feedback. This work proposes the first closed-loop, self-correcting multi-agent testing framework, in which three specialized agents—responsible for test generation, execution analysis, and review-based optimization—collaborate to enable feedback-driven iterative refinement. The framework innovatively integrates multi-agent collaboration with continuous learning, leveraging a sandboxed execution environment, fine-grained failure diagnostics, coverage-aware reinforcement signals, and a CI/CD-compatible pipeline to support automatic regeneration and repair of test cases. Evaluated on microservice applications, the approach reduces invalid tests by 60% and improves coverage by 30%, substantially decreasing the need for manual intervention.

This work addresses the lack of effective evaluation benchmarks for AI models in real-world software formal verification. It presents the first large-scale effort to automatically translate real-world Python property-based tests (PBT) into formal specifications in Lean 4, establishing a reproducible benchmark. By integrating a multi-agent large language model pipeline with Python semantic modeling and dependent type programming techniques, the authors successfully generated 9,415 Lean 4 specifications corresponding to 2,772 tests derived from 11,039 original PBTs. The entire codebase and dataset are publicly released. This contribution substantially advances the practical application of AI-assisted formal verification in real software systems.