Generative Goal Modeling

2025-08-31

AI Summary

In software engineering, manual extraction of goals from stakeholder interviews and subsequent goal modeling suffer from low efficiency and poor reproducibility. To address this, we propose the first end-to-end automated goal modeling method integrating textual entailment reasoning with a large language model (GPT-4o). Our approach directly generates structured goal models from unstructured interview transcripts, supporting high-level goal-to-software-operation mapping, requirement refinement, and conflict/obstacle analysis, while enabling goal provenance tracing and refinement relation inference. Evaluated on 15 cross-domain interview datasets, it achieves a goal matching rate of 62.0% (comparable to human performance), a provenance tracing accuracy of 98.7%, and a refinement relation generation accuracy of 72.2%. The core innovation lies in the first application of textual entailment to goal modeling, significantly enhancing both the accuracy and interpretability of automated goal modeling.

Technology Category

• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

In software engineering, requirements may be acquired from stakeholders through elicitation methods, such as interviews, observational studies, and focus groups. When supporting acquisition from interviews, business analysts must review transcripts to identify and document requirements. Goal modeling is a popular technique for representing early stakeholder requirements as it lends itself to various analyses, including refinement to map high-level goals into software operations, and conflict and obstacle analysis. In this paper, we describe an approach to use textual entailment to reliably extract goals from interview transcripts and to construct goal models. The approach has been evaluated on 15 interview transcripts across 29 application domains. The findings show that GPT-4o can reliably extract goals from interview transcripts, matching 62.0% of goals acquired by humans from the same transcripts, and that GPT-4o can trace goals to originating text in the transcript with 98.7% accuracy. In addition, when evaluated by human annotators, GPT-4o generates goal model refinement relationships among extracted goals with 72.2% accuracy.

Problem

• Extracting goals from interview transcripts automatically
• Constructing goal models using textual entailment techniques
• Evaluating GPT-4o's accuracy in goal identification and tracing

Innovation

• GPT-4o extracts goals from interview transcripts
• Textual entailment constructs goal models automatically
• Goal model refinement with high accuracy rates