Requirements-Based Test Generation: A Comprehensive Survey

2025-05-04

AI Summary

This study addresses the fundamental challenge of automatically generating test cases from user requirements—without access to source code—to ensure software quality and requirement consistency. We conduct the first systematic literature review (SLR) on Requirement-Based Test Generation (RBTG), synthesizing over three decades of research. Methodologically, we propose the first multidimensional RBTG taxonomy, encompassing requirement types, generation techniques, test forms, tool implementations, and evaluation mechanisms. We construct an RBTG knowledge graph, categorizing 12 requirement types, 7 method families, 23 tools, and 6 industrial application scenarios. Our analysis identifies critical challenges—including insufficient explainability and semantic gaps—and highlights promising future directions, such as large language model integration and standardization for industrial adoption. This work fills a significant gap in RBTG survey literature and provides foundational support for both theoretical advancement and practical deployment of RBTG.

Technology Category

• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data generation and simulation
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

As an important way of assuring software quality, software testing generates and executes test cases to identify software failures. Many strategies have been proposed to guide test-case generation, such as source-code-based approaches and methods based on bug reports. Requirements-based test generation (RBTG) constructs test cases based on specified requirements, aligning with user needs and expectations, without requiring access to the source code. Since its introduction in 1994, there have been many contributions to the development of RBTG, including various approaches, implementations, tools, assessment and evaluation methods, and applications. This paper provides a comprehensive survey on RBTG, categorizing requirement types, classifying approaches, investigating types of test cases, summarizing available tools, and analyzing experimental evaluations. This paper also summarizes the domains and industrial applications of RBTG, and discusses some open research challenges and potential future work.

Problem

• Surveying requirements-based test generation methods and tools
• Classifying approaches for test case generation from requirements
• Analyzing RBTG applications and identifying research challenges

Innovation

• Generates test cases from specified requirements
• Aligns testing with user needs and expectations
• Does not require access to source code