Conjecture and Inquiry: Quantifying Software Performance Requirements via Interactive Retrieval-Augmented Preference Elicitation

2026-04-23

AI Summary

This work addresses the challenge of automatically translating natural language descriptions of software performance requirements into precise mathematical formulations, a task often hindered by linguistic ambiguity and cognitive uncertainty. The authors propose an interactive, retrieval-augmented preference elicitation method that uniquely integrates domain-specific knowledge into both preference inference and dialogue guidance. By leveraging this knowledge to steer conversational interactions, the approach incrementally refines user intent into accurate mathematical functions. Evaluated on four real-world datasets, the method substantially outperforms ten state-of-the-art baselines, achieving up to a 40-fold improvement in performance with only five rounds of interaction. This significant gain demonstrates its effectiveness in reducing users’ cognitive load while enhancing the efficiency and precision of requirements engineering.

Technology Category

• Knowledge Representation and Reasoning: Preferences
• Machine Learning: Learning Preferences or Rankings
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Since software performance requirements are documented in natural language, quantifying them into mathematical forms is essential for software engineering. Yet, the vagueness in performance requirements and uncertainty of human cognition have caused highly uncertain ambiguity in the interpretations, rendering their automated quantification an unaddressed and challenging problem. In this paper, we formalize the problem and propose IRAP, an approach that quantifies performance requirements into mathematical functions via interactive retrieval-augmented preference elicitation. IRAP differs from the others in that it explicitly derives from problem-specific knowledge to retrieve and reason the preferences, which also guides the progressive interaction with stakeholders, while reducing the cognitive overhead. Experiment results against 10 state-of-the-art methods on four real-world datasets demonstrate the superiority of IRAP on all cases with up to 40x improvements under as few as five rounds of interactions.

Problem

• software performance requirements
• quantification
• ambiguity
• preference elicitation
• natural language

Innovation

• retrieval-augmented preference elicitation
• performance requirement quantification
• interactive elicitation
• software engineering
• mathematical formalization