[Exploring Dynamic Load Balancing Algorithms] {Exploring Dynamic Load Balancing Algorithms for Block-Structured Mesh-and-Particle Simulations in AMReX}

2025-05-21

AI Summary

This work addresses the challenge of dynamic load balancing for block-structured grid–particle coupled simulations within the AMReX framework on heterogeneous supercomputing systems. To tackle hardware-aware partitioning inefficiencies, we propose two topology-aware algorithms: (1) a hybrid partitioning scheme integrating the knapsack optimization with space-filling curves (SFCs), and (2) an SFC-based refinement inspired by painter’s partitioning. Compared to AMReX’s default percentage-based load balancing, our methods significantly improve balance quality—particularly under low-weight-variance scenarios. Rigorous evaluation via knapsack optimization, topology-aware domain decomposition, and brute-force enumeration benchmarks demonstrates superior partitioning quality and runtime efficiency across most test cases, with pronounced advantages when particle weight distributions are uniform. The proposed approaches establish a scalable, high-performance paradigm for dynamic load balancing in large-scale, multi-physics HPC simulations.

Technology Category

• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Machine Learning: Hardware-aware ML
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Load balancing is critical for successful large-scale high-performance computing (HPC) simulations. With modern supercomputers increasing in complexity and variability, dynamic load balancing is becoming more critical to use computational resources efficiently. In this study, performed during a summer collaboration at Lawrence Berkeley National Laboratory, we investigate various standard dynamic load-balancing algorithms. This includes the time evaluation of a brute-force solve for application in algorithmic evaluation, as well as quality and time evaluations of the Knapsack algorithm, an SFC algorithm, and two novel algorithms: a painter's partition-based SFC algorithm and a combination Knapsack+SFC methodology-based on hardware topology. The results suggest Knapsack and painter's partition-based algorithms should be among the first algorithms evaluated by HPC codes for cases with limited weight deviation and will perform at least slightly better than AMReX's percentage-tracking partitioning strategy across most simulations, although effects diminish as weight variety increases.

Problem

• Exploring dynamic load balancing for AMReX simulations
• Evaluating Knapsack and SFC algorithms for HPC efficiency
• Comparing novel partition-based methods for load balancing

Innovation

• Evaluates Knapsack algorithm for load balancing
• Introduces painter's partition-based SFC algorithm
• Proposes combined Knapsack and SFC methodology