The Role of Cognitive Abilities in Requirements Inspection: Comparing UML and Textual Representations

2026-01-22

AI Summary

This study investigates how the format of requirements representation—UML sequence diagrams versus plain text—affects the accuracy of requirements inspection, and examines the moderating roles of working memory capacity and mental rotation ability. Employing a crossover experimental design, the research integrates cognitive ability assessments with task performance data, analyzed via linear mixed-effects models and generalized linear models. Results reveal a significant three-way interaction among representation type and the two cognitive abilities: participants with higher cognitive capacities provided more accurate justifications for identified defects when supported by UML diagrams, yet paradoxically exhibited lower defect detection rates. These findings challenge the assumption that UML universally enhances inspection effectiveness and demonstrate that the benefits of multimodal representations are highly contingent on individual cognitive characteristics.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Cognitive Modeling & Cognitive Systems: Symbolic Representations
• Reasoning under Uncertainty: Uncertainty Representations

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

The representation of requirements plays a critical role in the accuracy of requirements inspection. While visual representations, such as UML diagrams, are widely used alongside text-based requirements, their effectiveness in supporting inspection is still debated. Cognitive abilities, such as working memory and mental rotation skills, may also influence inspection accuracy. This study aims to evaluate whether the use of UML sequence diagrams alongside text-based requirements improves the accuracy of requirements inspection compared to text-based requirements alone and to explore whether cognitive abilities are associated with differences in performance across the two treatments (text vs text with UML support). We conducted a crossover experiment with 38 participants to assess the accuracy of requirements inspection under the two treatments in terms of issues found and justifications provided. Linear mixed-effects and generalized linear models were used to analyse the effects of treatment, period, sequence, and cognitive abilities. The results indicate a significant three-way interaction between representation type, working memory capacity, and mental rotation ability. This finding suggests that the effectiveness of UML support is not uniform across individuals: participants with high scores in both cognitive abilities experienced reduced performance when using UML for violation detection. Conversely, the same cognitive profile was associated with improved justification accuracy under UML-aided inspection, indicating that higher cognitive abilities may support deeper reasoning processes when dealing with multi-modal information, i.e., diagrams and text.

Problem

• requirements inspection
• UML diagrams
• cognitive abilities
• working memory
• mental rotation

Innovation

• cognitive abilities
• requirements inspection
• UML diagrams
• working memory
• mental rotation