🤖 AI Summary
This study investigates key factors influencing analysts’ ability to identify domain elements—concepts, processes, and requirements—during requirements elicitation, focusing on their “visibility”: the degree to which such elements are accurately recognized and articulated. Employing 14 controlled experiments with 134 participants across two distinct problem domains, the research quantitatively analyzes visibility determinants. Results reveal that verbal rehearsal significantly enhances visibility, exposing a systematic gap between knowledge reportability and actual domain understanding; domain complexity, element criticality, and interview script design also exert significant effects, whereas analyst experience and client interaction intensity do not. Critically, this work provides the first empirical evidence that information presentation modality—particularly feedback modality—is the dominant determinant of visibility. These findings offer foundational theoretical insights and methodological guidance for improving requirements elicitation practices and requirements documentation quality.
📝 Abstract
Background: Various factors determine analyst effectiveness during elicitation. While the literature suggests that elicitation technique and time are influential factors, other attributes could also play a role. Aim: Determine aspects that may have an influence on analysts' ability to identify certain elements of the problem domain. Methodology: We conducted 14 quasi-experiments, inquiring 134 subjects about two problem domains. For each problem domain, we calculated whether the experimental subjects identified the problem domain elements (concepts, processes, and requirements), i.e., the degree to which these domain elements were visible. Results: Domain element visibility does not appear to be related to either analyst experience or analyst-client interaction. Domain element visibility depends on how analysts provide the elicited information: when asked about the knowledge acquired during elicitation, domain element visibility dramatically increases compared to the information they provide using a written report. Conclusions: Further research is required to replicate our results. However, the finding that analysts have difficulty reporting the information they have acquired is useful for identifying alternatives for improving the documentation of elicitation results. We found evidence that other issues, like domain complexity, the relative importance of different elements within the domain, and the interview script, also seem influential.