Graph Neural Networks Gone Hogwild

2024-06-29

International Conference on Learning Representations

AI Summary

Graph neural networks (GNNs) suffer from catastrophic prediction failure under asynchronous inference in distributed multi-agent systems—severely limiting their deployment in resource-constrained, inherently desynchronized settings such as robot swarms and sensor networks. To address this, we first establish the first theoretical robustness guarantee for message-passing GNNs under partially asynchronous (Hogwild-style) inference. We then propose an energy-driven implicit GNN architecture: it formulates message aggregation via an energy function and solves representations through provably convergent fixed-point iteration. This design ensures both theoretical convergence and strong empirical performance. On multi-agent synthetic tasks, our method significantly outperforms existing implicit GNNs; on real-world graph benchmarks, it achieves competitive accuracy. Our work thus bridges theory and practice by providing formal robustness guarantees for asynchronous GNN inference while delivering state-of-the-art performance across diverse evaluation scenarios.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Multiagent Systems: Adversarial Agents
• Natural Language Processing: Safety and Robustness

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Message passing graph neural networks (GNNs) would appear to be powerful tools to learn distributed algorithms via gradient descent, but generate catastrophically incorrect predictions when nodes update asynchronously during inference. This failure under asynchrony effectively excludes these architectures from many potential applications, such as learning local communication policies between resource-constrained agents in, e.g., robotic swarms or sensor networks. In this work we explore why this failure occurs in common GNN architectures, and identify"implicitly-defined"GNNs as a class of architectures which is provably robust to partially asynchronous"hogwild"inference, adapting convergence guarantees from work in asynchronous and distributed optimization, e.g., Bertsekas (1982); Niu et al. (2011). We then propose a novel implicitly-defined GNN architecture, which we call an energy GNN. We show that this architecture outperforms other GNNs from this class on a variety of synthetic tasks inspired by multi-agent systems, and achieves competitive performance on real-world datasets.

Problem

• GNNs fail under asynchronous node updates during inference
• Asynchrony limits GNN applications in decentralized multi-agent systems
• Proposing robust implicitly-defined GNNs for asynchronous environments

Innovation

• Implicitly-defined GNNs ensure asynchronous robustness
• Energy GNN architecture outperforms traditional GNNs
• Adapts optimization guarantees for hogwild inference