Requirements Elicitation Follow-Up Question Generation

2025-07-03

AI Summary

During requirements elicitation interviews, experienced interviewers often struggle to formulate high-quality follow-up questions in real time due to domain knowledge gaps, high cognitive load, and information overload. Method: We propose an error-guided prompting framework that integrates a taxonomy of common interviewer errors with the GPT-4o large language model, enabling dynamic generation of contextually relevant questions grounded in受访者 utterances. Structured prompting enhances question relevance and informativeness. Contribution/Results: Empirical evaluation shows that LLM-generated questions match human-authored questions in clarity, relevance, and informativeness—and significantly outperform baseline LLM approaches when guided by error categories. Our approach establishes an interpretable, reusable generative paradigm for intelligent requirements engineering assistance, effectively alleviating interviewer cognitive burden while improving both the quality and efficiency of requirements elicitation.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Interviews are a widely used technique in eliciting requirements to gather stakeholder needs, preferences, and expectations for a software system. Effective interviewing requires skilled interviewers to formulate appropriate interview questions in real time while facing multiple challenges, including lack of familiarity with the domain, excessive cognitive load, and information overload that hinders how humans process stakeholders' speech. Recently, large language models (LLMs) have exhibited state-of-the-art performance in multiple natural language processing tasks, including text summarization and entailment. To support interviewers, we investigate the application of GPT-4o to generate follow-up interview questions during requirements elicitation by building on a framework of common interviewer mistake types. In addition, we describe methods to generate questions based on interviewee speech. We report a controlled experiment to evaluate LLM-generated and human-authored questions with minimal guidance, and a second controlled experiment to evaluate the LLM-generated questions when generation is guided by interviewer mistake types. Our findings demonstrate that, for both experiments, the LLM-generated questions are no worse than the human-authored questions with respect to clarity, relevancy, and informativeness. In addition, LLM-generated questions outperform human-authored questions when guided by common mistakes types. This highlights the potential of using LLMs to help interviewers improve the quality and ease of requirements elicitation interviews in real time.

Problem

• Generating real-time follow-up questions for requirements elicitation interviews
• Addressing challenges like domain unfamiliarity and cognitive load in interviews
• Evaluating LLM-generated questions versus human-authored questions for quality

Innovation

• Uses GPT-4o to generate follow-up questions
• Applies LLMs for real-time interview support
• Guides question generation by mistake types