🤖 AI Summary
Existing advanced test case generation approaches heavily rely on manual effort or customized RAG systems, suffering from poor generalizability and high construction costs.
Method: This paper proposes a pure prompt-engineering–driven large language model (LLM) approach that automatically generates functional-level natural language test cases directly from requirements documents—without retrieval-augmented generation (RAG). It first guides the LLM to identify appropriate test design techniques for the given requirements, then leverages this selection to generate concrete test cases, enabling end-to-end zero-shot inference.
Contribution/Results: The method significantly improves cross-domain generalization and industrial deployability. On the Bluetooth and Mozilla datasets, it achieves macro-recall scores of 0.81 and 0.37, respectively, demonstrating both effectiveness and practical potential. To our knowledge, this is the first work achieving high-level test case automation solely through structured prompting—without any retrieval augmentation.
📝 Abstract
Currently, generating high-level test cases described in natural language from requirement documents is performed manually. In the industry, including companies specializing in software testing, there is a significant demand for the automatic generation of high-level test cases from requirement documents using Large Language Models (LLMs). Efforts to utilize LLMs for requirement analysis are underway. In some cases, retrieval-augmented generation (RAG) is employed for generating high-level test cases using LLMs. However, in practical applications, it is necessary to create a RAG tailored to the knowledge system of each specific application, which is labor-intensive. Moreover, when applying high-level test case generation as a prompt, there is no established method for instructing the generation of high-level test cases at a level applicable to other specifications without using RAG. It is required to establish a method for the automatic generation of high-level test cases that can be generalized across a wider range of requirement documents. In this paper, we propose a method for generating high-level (GHL) test cases from requirement documents using only prompts, without creating RAGs. In the proposed method, first, the requirement document is input into the LLM to generate test design techniques corresponding to the requirement document. Then, high-level test cases are generated for each of the generated test design techniques. Furthermore, we verify an evaluation method based on semantic similarity of the generated high-level test cases. In the experiments, we confirmed the method using datasets from Bluetooth and Mozilla, where requirement documents and high-level test cases are available, achieving macro-recall measurement of 0.81 and 0.37, respectively. We believe that the method is feasible for practical application in generating high-level test cases without using RAG.