Machine Learning as a Service (MLaaS) Dataset Generator Framework for IoT Environments

2026-01-18

AI Summary

This work addresses the lack of configurable and reproducible benchmark datasets for evaluating Machine Learning as a Service (MLaaS) systems. To bridge this gap, the authors propose MDG, the first framework for generating MLaaS datasets tailored to IoT environments. MDG simulates realistic MLaaS behavior by training diverse model families across multiple real-world datasets and distribution settings, while systematically capturing functional attributes, Quality of Service (QoS) metrics, and service composition indicators. The framework enables large-scale, extensible benchmark construction and incorporates built-in support for service composition modeling and IoT scenario simulation. Experimental results demonstrate that MDG generates over 10,000 MLaaS service instances, significantly improving the accuracy of service selection and the quality of service compositions, thereby establishing a reliable foundation for MLaaS research and evaluation.

Technology Category

• Machine Learning: Scalability of ML Systems
• Humans and AI: Human-in-the-loop Machine Learning
• Multiagent Systems: Applications

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We propose a novel MLaaS Dataset Generator (MDG) framework that creates configurable and reproducible datasets for evaluating Machine Learning as a Service (MLaaS) selection and composition. MDG simulates realistic MLaaS behaviour by training and evaluating diverse model families across multiple real-world datasets and data distribution settings. It records detailed functional attributes, quality of service metrics, and composition-specific indicators, enabling systematic analysis of service performance and cross-service behaviour. Using MDG, we generate more than ten thousand MLaaS service instances and construct a large-scale benchmark dataset suitable for downstream evaluation. We also implement a built-in composition mechanism that models how services interact under varied Internet of Things conditions. Experiments demonstrate that datasets generated by MDG enhance selection accuracy and composition quality compared to existing baselines. MDG provides a practical and extensible foundation for advancing data-driven research on MLaaS selection and composition

Problem

• Machine Learning as a Service
• Dataset Generation
• IoT Environments
• Service Composition
• Benchmarking

Innovation

• MLaaS
• Dataset Generation
• Service Composition
• IoT
• Benchmarking