Profiling Apple Silicon Performance for ML Training

2025-01-24

AI Summary

This study systematically evaluates performance disparities between Apple Silicon (M-series) and NVIDIA GPUs in end-to-end large language model (LLM) training and fundamental linear algebra computations. Method: We establish a cross-layer analytical framework integrating unified memory behavior monitoring, system-level profiling (via perf/Instruments), BLAS library benchmarking, and GPU kernel launch latency measurement. Contribution/Results: For the first time, we attribute performance bottlenecks to three root causes: excessive page faults saturating memory bandwidth, sustained frequency throttling under energy constraints, and high Metal kernel launch overhead. Empirical results demonstrate that Apple Silicon significantly underperforms comparable NVIDIA GPUs in mainstream LLM training. We propose the first unified performance explanation model for heterogeneous AI training platforms—grounded in empirical evidence—and provide actionable insights and methodological foundations for ARM-based AI accelerator design and optimization.

Technology Category

• Machine Learning: Hardware-aware ML
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Apple Silicon has attracted much attention for its performance and role in machine learning (ML) training. Unlike NVIDIA GPUs, which have traditionally dominated ML training, Apple Silicon has a significant difference in memory architecture. It uses Unified Memory, which integrates CPU and GPU memory instead of separate CPU memory and GPU VRAM. However, it is difficult to tell whether Unified Memory means more performance benefits. This paper investigates the performance differences by training several large language model (LLM) workloads end-to-end under different memory scenarios. The results show a significant performance gap between Apple Silicon and NVIDIA GPUs. This paper attributes this gap to system-level factors such as page faults, power consumption, and kernel launch time. In addition, the performance difference of basic linear algebra subprograms (BLAS) on the NVIDIA GPUs and Apple Silicon chips is analyzed to further explain the observed gap.

Problem

• Apple Silicon
• NVIDIA GPU
• Machine Learning Performance

Innovation

• Apple Silicon
• Machine Learning Performance
• Unified Memory Design