Comparing Cross-Platform Performance via Node-to-Node Scaling Studies

2025-10-14

AI Summary

A lack of systematic methodologies for cross-platform performance and scalability comparison across heterogeneous HPC systems hinders fair and reproducible evaluation. Method: This paper proposes a unified cross-platform evaluation paradigm centered on the single compute node as the baseline unit. It integrates node-level performance measurement, weak/strong scaling analysis, and normalized metric comparison into a standardized experimental design, execution, and reporting workflow. A general-purpose validation framework is developed and empirically applied across diverse architectures—including CPUs, GPUs, and heterogeneous accelerators. Contributions/Results: (1) Establishes the single node as the minimal comparable unit for cross-platform assessment; (2) Provides a reusable, standardized evaluation template and integrated toolchain; (3) Significantly improves consistency, reproducibility, and interpretability of performance evaluation across heterogeneous HPC platforms.

Technology Category

• Machine Learning: Hardware-aware ML
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Humans and AI: Other Foundations of Human Computation & AI

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

Due to the increasing diversity of high-performance computing architectures, researchers and practitioners are increasingly interested in comparing a code's performance and scalability across different platforms. However, there is a lack of available guidance on how to actually set up and analyze such cross-platform studies. In this paper, we contend that the natural base unit of computing for such studies is a single compute node on each platform and offer guidance in setting up, running, and analyzing node-to-node scaling studies. We propose templates for presenting scaling results of these studies and provide several case studies highlighting the benefits of this approach.

Problem

• Comparing code performance across diverse HPC architectures
• Lack of guidance for cross-platform performance analysis setup
• Providing node-to-node scaling methodology for platform comparison

Innovation

• Node-to-node scaling as base comparison unit
• Templates for presenting cross-platform scaling results
• Guidance for setup and analysis of performance studies