AI-Assisted Requirements Engineering: An Empirical Evaluation Relative to Expert Judgment

2026-04-16

AI Summary

This study addresses the unclear role of current AI tools in requirements engineering and their alignment with INCOSE standards. Designing a controlled experiment grounded in INCOSE’s “good requirements” criteria, the work systematically compares AI tools and human experts across key quality dimensions—consistency, completeness, clarity, and testability. The findings empirically demonstrate, for the first time, that AI can efficiently and consistently perform initial quality assessments at the syntactic and structural levels, yet still requires expert intervention for higher-order tasks involving contextual understanding, ambiguity resolution, and trade-off reasoning. The results delineate a clear integration pathway wherein AI serves as a decision-support aid rather than a replacement for human judgment, offering both methodological foundations and practical guidance for AI-assisted requirements validation.

Technology Category

• Constraint Satisfaction and Optimization: Solvers and Tools
• Humans and AI: User Experience and Usability
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content

Abstract

Artificial Intelligence is increasingly introduced into systems engineering activities, particularly within requirements engineering, where quality assessment and validation remain heavily dependent on expert judgment. While recent AI tools demonstrate promising capabilities in analyzing and generating requirements, their role within formal systems engineering processes-and their alignment with established INCOSE criteria-remains insufficiently understood. This paper investigates the extent to which AI-based tools can support systems engineers in evaluating requirement quality, without replacing professional expertise. The research adopts a structured systems engineering methodology to compare AI-assisted requirement evaluation with human expert assessment. A controlled study was conducted in which system requirements were evaluated against established INCOSE ``good requirement'' criteria by both experienced systems engineers and an AI-based assessment tool. The evaluation focused on consistency, completeness, clarity, and testability, examining not only accuracy but also the decision logic underlying each assessment. Results indicate that AI tools can provide consistent and rapid preliminary assessments, particularly for syntactic and structural quality attributes. However, expert judgment remains essential for contextual interpretation, ambiguity resolution, and trade-off reasoning. Rather than positioning AI as a replacement for systems engineers, the findings support its role as a decision-support mechanism within the RE lifecycle. From a systems engineering perspective, this study contributes empirical evidence on how AI can be integrated into RE workflows while preserving traceability, accountability, and engineering consistency.

Problem

• Requirements Engineering
• Artificial Intelligence
• Expert Judgment
• INCOSE Criteria
• Requirement Quality

Innovation

• AI-assisted requirements engineering
• empirical evaluation
• INCOSE criteria
• decision support
• requirement quality assessment