From Failure to Alignment: A Requirements Engineering Framework for Machine Learning Systems

2026-06-30

AI Summary

This work addresses the challenge of ensuring trustworthiness and stakeholder alignment in machine learning system development, which is often hindered by the absence of systematic requirements engineering. To bridge this gap, the authors propose REAL, a novel framework that uniquely integrates failure mode analysis into the requirements engineering process. REAL establishes a tripartite principle centered on data, model, and holistic system requirements, enabling iterative and traceable requirement refinement. Through a model-driven, stakeholder-oriented design, REAL demonstrates substantial improvements in requirement satisfaction in an autonomous driving case study. The authors further support reproducibility by releasing an open-source implementation toolkit.

Abstract

Organisations designing, developing, and deploying machine learning systems (MLS) need to be able to check that these systems are trustworthy, and communicate this clearly to their stakeholders, be they different categories of users, engineers, or wider society. By focusing on stakeholders, Requirements Engineering is well positioned to drive the design and engineering of MLS that align with the needs of their stakeholders. Yet, we still need a systematic process for modelling and reasoning about requirements for MLS that is driven both by stakeholders' needs and constraints for MLS development. This paper proposes a framework entitled REAL (Requirements Engineering for mAchines that Learn - and Fail) to help develop MLS that align with stakeholders' needs by adopting a requirements engineering approach. This model-based framework is based on three principles. First, weaving together requirements for data, models, and the system as a whole. Second, using failure to drive the exploration of alternative requirements. Third, iterative and traceable refinement of MLS requirements. We demonstrate the proposed framework using an example from autonomous driving and show that REAL supports the development of MLS that better align with stakeholders' requirements. A replication package is available online.

Problem

• Requirements Engineering
• Machine Learning Systems
• Stakeholder Alignment
• Trustworthiness
• System Failure

Innovation

• Requirements Engineering
• Machine Learning Systems
• Failure-driven Design
• Stakeholder Alignment
• Model-based Framework