Survey on Characterizing and Understanding GNNs from a Computer Architecture Perspective

2024-08-04

AI Summary

Graph Neural Networks (GNNs) suffer from low execution efficiency and poor training stability on parallel and distributed systems due to unoptimized computational, memory access, and communication patterns. Method: This paper adopts a computer architecture–centric perspective to systematically analyze these patterns and identify critical performance bottlenecks. It proposes the first hardware-software co-design–oriented, three-level GNN workload taxonomy—unifying architectural characterization across diverse GNN models. Leveraging architecture-aware analysis, fine-grained performance modeling, and cross-platform empirical evaluation, it quantifies behavioral disparities of GNN workloads across CPUs, GPUs, and distributed clusters. Contribution/Results: The work delivers a reusable modeling framework and actionable optimization guidelines for efficient, architecture-agnostic GNN implementation. It significantly improves hardware resource utilization and training stability while enabling systematic, principled design of high-performance GNN systems.

Technology Category

• Machine Learning: Hardware-aware ML
• Cognitive Modeling & Cognitive Systems: (Computational) Cognitive Architectures
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Social Networks and Social Media: Computational social science

Abstract

Characterizing and understanding graph neural networks (GNNs) is essential for identifying performance bottlenecks and facilitating their deployment in parallel and distributed systems. Despite substantial work in this area, a comprehensive survey on characterizing and understanding GNNs from a computer architecture perspective is lacking. This work presents a comprehensive survey, proposing a triple-level classification method to categorize, summarize, and compare existing efforts, particularly focusing on their implications for parallel architectures and distributed systems. We identify promising future directions for GNN characterization that align with the challenges of optimizing hardware and software in parallel and distributed systems. Our survey aims to help scholars systematically understand GNN performance bottlenecks and execution patterns from a computer architecture perspective, thereby contributing to the development of more efficient GNN implementations across diverse parallel architectures and distributed systems.

Problem

• Graph Neural Networks
• Performance Optimization
• Parallel Processing

Innovation

• Graph Neural Networks
• Hardware Design
• Parallel and Distributed Systems