Applying a Requirements-Focused Agile Management Approach for Machine Learning-Enabled Systems

2026-02-04

AI Summary

This study addresses the limitations of traditional requirements engineering and agile methodologies in handling the data dependency, experimental nature, and model uncertainty inherent in machine learning (ML) systems, as well as their insufficient integration of ML-specific characteristics. To bridge this gap, the authors propose and evaluate RefineML—a framework developed in collaboration between PUC-Rio and EXA, a Brazilian cybersecurity company—that integrates ML-tailored requirements specification with agile management to enable co-evolution of models and software. The framework was informed by a systematic mapping study to identify best practices and empirically assessed through questionnaires, semi-structured interviews, and thematic analysis. Findings indicate that RefineML significantly enhances cross-role communication and supports early feasibility assessment and continuous refinement, yet challenges remain in operationalizing ML requirements and estimating associated effort.

Technology Category

• Machine Learning: Scalability of ML Systems
• Humans and AI: Human-in-the-loop Machine Learning
• Multiagent Systems: Mechanism Design

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations

Abstract

Machine Learning (ML)-enabled systems challenge traditional Requirements Engineering (RE) and agile management due to data dependence, experimentation, and uncertain model behavior. Existing RE and agile practices remain poorly integrated and insufficiently tailored to these characteristics. This paper reports on the practical experience of applying RefineML, a requirements-focused approach for the continuous and agile refinement of ML-enabled systems, which integrates ML-tailored specification and agile management approaches with best practices derived from a systematic mapping study. The application context concerns an industry-academia collaboration project between PUC-Rio and EXA, a Brazilian cybersecurity company. For evaluation purposes, we applied questionnaires assessing RefineML's suitability and overall acceptance and semi-structured interviews. We applied thematic analysis to the collected qualitative data. Regarding suitability and acceptance, the results of the questionnaires indicated high perceived usefulness and intention to use. Based on the interviews, stakeholders perceived RefineML as improving communication and facilitating early feasibility assessments, as well as enabling dual-track governance of ML and software work, allowing continuous refinement of the model while evolving the overall software project. However, some limitations remain, particularly related to difficulties in operationalizing ML concerns into agile requirements and in estimating ML effort.

Problem

• Requirements Engineering
• Agile Management
• Machine Learning-enabled Systems
• Data Dependence
• Model Uncertainty

Innovation

• RefineML
• requirements engineering
• agile management
• dual-track governance
• ML-enabled systems