Define-ML: An Approach to Ideate Machine Learning-Enabled Systems

2025-06-25

AI Summary

Traditional Lean Inception and similar ideation methods lack structured support for ML-specific concerns—namely data dependencies, technical feasibility, and alignment between business objectives and probabilistic system behavior—leading to vision drift and misaligned expectations. Method: We propose Define-ML, an extension of Lean Inception incorporating three novel activities: data source mapping, feature–data source mapping, and ML mapping. This explicitly embeds data constraints and model capabilities into early product ideation. The framework was developed and validated via the Technology Transfer Model, using both a controlled toy problem (static) and an industrial case study (dynamic), supplemented by surveys and expert interviews. Contribution/Results: Define-ML significantly improves cross-functional alignment, clarifies data bottlenecks, and reduces ideation ambiguity. All participants confirmed willingness to adopt it; expert facilitation effectively mitigates implementation barriers.

Technology Category

• Machine Learning: Active Learning
• Humans and AI: Human-in-the-loop Machine Learning
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

[Context] The increasing adoption of machine learning (ML) in software systems demands specialized ideation approaches that address ML-specific challenges, including data dependencies, technical feasibility, and alignment between business objectives and probabilistic system behavior. Traditional ideation methods like Lean Inception lack structured support for these ML considerations, which can result in misaligned product visions and unrealistic expectations. [Goal] This paper presents Define-ML, a framework that extends Lean Inception with tailored activities - Data Source Mapping, Feature-to-Data Source Mapping, and ML Mapping - to systematically integrate data and technical constraints into early-stage ML product ideation. [Method] We developed and validated Define-ML following the Technology Transfer Model, conducting both static validation (with a toy problem) and dynamic validation (in a real-world industrial case study). The analysis combined quantitative surveys with qualitative feedback, assessing utility, ease of use, and intent of adoption. [Results] Participants found Define-ML effective for clarifying data concerns, aligning ML capabilities with business goals, and fostering cross-functional collaboration. The approach's structured activities reduced ideation ambiguity, though some noted a learning curve for ML-specific components, which can be mitigated by expert facilitation. All participants expressed the intention to adopt Define-ML. [Conclusion] Define-ML provides an openly available, validated approach for ML product ideation, building on Lean Inception's agility while aligning features with available data and increasing awareness of technical feasibility.

Problem

• Addresses ML-specific challenges in software system ideation
• Extends Lean Inception with structured ML-focused activities
• Aligns ML capabilities with business goals and data constraints

Innovation

• Extends Lean Inception with ML-specific activities
• Integrates data constraints into early ideation
• Validated via static and dynamic methods