Causal-Aware Graph Neural Architecture Search under Distribution Shifts

2024-05-26

arXiv.org

AI Summary

To address the weak out-of-distribution (OOD) generalization of Graph Neural Networks (GNNs) under distributional shifts, this paper proposes the first Graph Neural Architecture Search (GNAS) framework integrating causal discovery and intervention. Rather than modeling spurious correlations between graph structures and architecture performance, our approach explicitly models their causal relationships to mitigate unreliable structure-performance associations induced by distribution shifts. Methodologically, we introduce three novel mechanisms: (1) causal subgraph identification to disentangle structural factors critical to architecture performance; (2) causal intervention in the graph embedding space to eliminate confounding bias; and (3) differentiable architecture customization grounded in invariant learning. Evaluated on multiple OOD graph benchmarks, our method achieves an average accuracy improvement of 12.7% over state-of-the-art methods, demonstrating substantial gains in cross-domain generalization.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Graph NAS has emerged as a promising approach for autonomously designing GNN architectures by leveraging the correlations between graphs and architectures. Existing methods fail to generalize under distribution shifts that are ubiquitous in real-world graph scenarios, mainly because the graph-architecture correlations they exploit might be spurious and varying across distributions. We propose to handle the distribution shifts in the graph architecture search process by discovering and exploiting the causal relationship between graphs and architectures to search for the optimal architectures that can generalize under distribution shifts. The problem remains unexplored with following challenges: how to discover the causal graph-architecture relationship that has stable predictive abilities across distributions, and how to handle distribution shifts with the discovered causal graph-architecture relationship to search the generalized graph architectures. To address these challenges, we propose Causal-aware Graph Neural Architecture Search (CARNAS), which is able to capture the causal graph-architecture relationship during the architecture search process and discover the generalized graph architecture under distribution shifts. Specifically, we propose Disentangled Causal Subgraph Identification to capture the causal subgraphs that have stable prediction abilities across distributions. Then, we propose Graph Embedding Intervention to intervene on causal subgraphs within the latent space, ensuring that these subgraphs encapsulate essential features for prediction while excluding non-causal elements. Additionally, we propose Invariant Architecture Customization to reinforce the causal invariant nature of the causal subgraphs, which are utilized to tailor generalized graph architectures. Extensive experiments demonstrate that CARNAS achieves advanced out-of-distribution generalization ability.

Problem

• Causal Reasoning
• Graph Neural Networks
• Data Distribution Shift

Innovation

• CARNAS
• Causal Relationship
• Adaptive Graph Neural Network Structure