LLM-Driven Cost-Effective Requirements Change Impact Analysis

2025-10-31

AI Summary

Requirement Change Impact Analysis (CIA) is critical in software engineering but suffers from high manual effort and error-proneness. This paper proposes ProReFiCIA, a prompt-engineering-driven approach that leverages multi-LLM collaborative evaluation for efficient, low-cost automated CIA—without fine-tuning or domain-specific annotations. It employs structured, domain-customized prompts to guide large language models in accurately identifying cross-requirement impact relationships. Experiments on both benchmark and industrial datasets demonstrate a recall of 93.3%–95.8%, substantially outperforming existing baselines; moreover, the proportion of requirements requiring expert review is reduced to only 2.1%–8.5%, significantly alleviating expert workload. To our knowledge, this work is the first to systematically validate the robustness and practicality of prompt-based LLMs for industrial-scale CIA. It establishes a deployable paradigm for requirement evolution management, bridging the gap between LLM capabilities and real-world software engineering needs.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Requirements are inherently subject to changes throughout the software development lifecycle. Within the limited budget available to requirements engineers, manually identifying the impact of such changes on other requirements is both error-prone and effort-intensive. That might lead to overlooked impacted requirements, which, if not properly managed, can cause serious issues in the downstream tasks. Inspired by the growing potential of large language models (LLMs) across diverse domains, we propose ProReFiCIA, an LLM-driven approach for automatically identifying the impacted requirements when changes occur. We conduct an extensive evaluation of ProReFiCIA using several LLMs and prompts variants tailored to this task. Using the best combination of an LLM and a prompt variant, ProReFiCIA achieves a recall of 93.3% on a benchmark dataset and 95.8% on a newly created industry dataset, demonstrating its strong effectiveness in identifying impacted requirements. Further, the cost of applying ProReFiCIA remains small, as the engineer only needs to review the generated results, which represent between 2.1% and 8.5% of the entire set of requirements.

Problem

• Automating requirements change impact analysis using LLMs
• Reducing manual effort in identifying affected software requirements
• Achieving high recall with low-cost automated impact assessment

Innovation

• LLM-driven automated requirements change impact analysis
• ProReFiCIA uses tailored LLM and prompt combinations
• Cost-effective approach with high recall on benchmark datasets