Performance Measurements in the AI-Centric Computing Continuum Systems

2025-06-28

AI Summary

Traditional performance metrics for AI-driven Distributed Computing Continua (DCC) lag behind the demands of generative AI and large language models. This paper identifies structural gaps in existing cloud-edge-device co-design metric frameworks—particularly concerning sustainability, energy efficiency, and system observability. Through a systematic literature review and cross-layer workload analysis, we propose a multidimensional, AI-centric metric framework that introduces four novel dimensions: energy-efficiency ratio, carbon-aware latency, resource coordination rate, and observability entropy, along with principled criteria for metric selection. Moving beyond conventional throughput- and latency-centric paradigms, our framework provides both theoretical foundations and practical guidance for designing and evaluating DCC systems. It advances distributed AI toward greater efficiency, environmental sustainability, and interpretability.

Technology Category

• Humans and AI: Other Foundations of Human Computation & AI
• Machine Learning: Hardware-aware ML
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Over the Eight decades, computing paradigms have shifted from large, centralized systems to compact, distributed architectures, leading to the rise of the Distributed Computing Continuum (DCC). In this model, multiple layers such as cloud, edge, Internet of Things (IoT), and mobile platforms work together to support a wide range of applications. Recently, the emergence of Generative AI and large language models has further intensified the demand for computational resources across this continuum. Although traditional performance metrics have provided a solid foundation, they need to be revisited and expanded to keep pace with changing computational demands and application requirements. Accurate performance measurements benefit both system designers and users by supporting improvements in efficiency and promoting alignment with system goals. In this context, we review commonly used metrics in DCC and IoT environments. We also discuss emerging performance dimensions that address evolving computing needs, such as sustainability, energy efficiency, and system observability. We also outline criteria and considerations for selecting appropriate metrics, aiming to inspire future research and development in this critical area.

Problem

• Revisiting traditional performance metrics for AI-centric distributed computing
• Addressing evolving computing needs like sustainability and energy efficiency
• Establishing criteria for selecting metrics in distributed computing continuum

Innovation

• Review traditional and emerging performance metrics
• Focus on sustainability and energy efficiency
• Enhance system observability in DCC