Formal and Empirical Study of Metadata-Based Profiling for Resource Management in the Computing Continuum

2025-04-29

AI Summary

Resource management across the IoT/Edge/Cloud continuum faces a fundamental trade-off among prediction accuracy, real-time responsiveness, and deployment lightweightness; existing approaches rely either on runtime sampling or static rules, failing to reconcile these requirements. This paper proposes a metadata-driven runtime workload profiling framework. First, it formally defines the mapping between static metadata and dynamic runtime behavior. Second, it introduces a clustering-based mechanism for extracting semantically significant metadata features—without requiring execution traces or online profiling. Third, it integrates lightweight feature engineering with regression modeling to achieve low-overhead, high-accuracy resource demand prediction. Evaluated across Alibaba’s ML workloads and Google’s cluster traces, the framework maintains high prediction accuracy even under partial data anonymization, substantially outperforming conventional static prediction and online profiling baselines. It achieves superior real-time performance, prediction fidelity, and practical deployability.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Metareasoning and Metaheuristics
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

We present and formalize a general approach for profiling workload by leveraging only a priori available static metadata to supply appropriate resource needs. Understanding the requirements and characteristics of a workload's runtime is essential. Profiles are essential for the platform (or infrastructure) provider because they want to ensure that Service Level Agreements and their objectives (SLOs) are fulfilled and, at the same time, avoid allocating too many resources to the workload. When the infrastructure to manage is the computing continuum (i.e., from IoT to Edge to Cloud nodes), there is a big problem of placement and tradeoff or distribution and performance. Still, existing techniques either rely on static predictions or runtime profiling, which are proven to deliver poor performance in runtime environments or require laborious mechanisms to produce fast and reliable evaluations. We want to propose a new approach for it. Our profile combines the information from past execution traces with the related workload metadata, equipping an infrastructure orchestrator with a fast and precise association of newly submitted workloads. We differentiate from previous works because we extract the profile group metadata saliency from the groups generated by grouping similar runtime behavior. We first formalize its functioning and its main components. Subsequently, we implement and empirically analyze our proposed technique on two public data sources: Alibaba cloud machine learning workloads and Google cluster data. Despite relying on partially anonymized or obscured information, the approach provides accurate estimates of workload runtime behavior in real-time.

Problem

• Profiling workload using static metadata for resource allocation
• Optimizing resource distribution in IoT-Edge-Cloud computing continuum
• Combining past execution traces with metadata for fast workload association

Innovation

• Leverages static metadata for workload profiling
• Combines past execution traces with metadata
• Groups similar runtime behavior for saliency