Rapid training of Hamiltonian graph networks without gradient descent

2025-06-06

AI Summary

Traditional gradient-descent-based training of Hamiltonian Graph Networks (HGNs) suffers from slow convergence and poor generalization, yet physical system modeling demands strict preservation of Hamiltonian structure, geometric symmetries, and conservation laws. Method: We propose the first gradient-free training paradigm for HGNs, replacing iterative optimizers with an analytical parameter construction scheme based on random Fourier features and explicit encoding of permutation, rotation, and translation invariances. Contribution/Results: Our method achieves up to 600× speedup over standard optimization while matching the accuracy of Adam and 14 other optimizers. It rigorously enforces physical invariances on N-body and 3D mass-spring systems. Moreover, it demonstrates, for the first time, zero-shot cross-scale generalization—from 8 to 4096 nodes—thereby challenging and expanding the fundamental boundaries of training paradigms for physics-informed neural networks.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Learning with Manifolds
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Learning dynamical systems that respect physical symmetries and constraints remains a fundamental challenge in data-driven modeling. Integrating physical laws with graph neural networks facilitates principled modeling of complex N-body dynamics and yields accurate and permutation-invariant models. However, training graph neural networks with iterative, gradient-based optimization algorithms (e.g., Adam, RMSProp, LBFGS) often leads to slow training, especially for large, complex systems. In comparison to 15 different optimizers, we demonstrate that Hamiltonian Graph Networks (HGN) can be trained up to 600x faster--but with comparable accuracy--by replacing iterative optimization with random feature-based parameter construction. We show robust performance in diverse simulations, including N-body mass-spring systems in up to 3 dimensions with different geometries, while retaining essential physical invariances with respect to permutation, rotation, and translation. We reveal that even when trained on minimal 8-node systems, the model can generalize in a zero-shot manner to systems as large as 4096 nodes without retraining. Our work challenges the dominance of iterative gradient-descent-based optimization algorithms for training neural network models for physical systems.

Problem

• Fast training of Hamiltonian graph networks without gradients
• Learning dynamical systems with physical symmetries and constraints
• Generalizing from small to large systems without retraining

Innovation

• Replaces gradient descent with random feature construction
• Ensures physical symmetries via Hamiltonian Graph Networks
• Generalizes from small to large systems without retraining