Learn to Jump: Adaptive Random Walks for Long-Range Propagation through Graph Hierarchies

2025-09-01

AI Summary

Graph neural networks (GNNs) are fundamentally limited in modeling long-range dependencies due to their local message-passing mechanism, which constrains the theoretical receptive field. To address this, we propose Hierarchical Adaptive Random Walks (HARW), a novel framework that constructs a hierarchical graph structure and introduces learnable transition probabilities to dynamically switch between walks on the original graph and hierarchical shortcut edges. This enables effective propagation beyond the conventional receptive field limit. Our key innovation lies in tightly coupling hierarchical abstraction with adaptive random walks, allowing the model to reach distant node pairs in significantly fewer steps. On synthetic long-range dependency benchmarks, HARW achieves comparable or superior performance to standard long walks on the original graph—using only approximately one-third of the walk steps. This yields substantial gains in inference efficiency and propagation range for large-scale graphs, establishing a new paradigm for modeling long-range dependencies in GNNs.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Graphical Models
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Message-passing architectures struggle to sufficiently model long-range dependencies in node and graph prediction tasks. We propose a novel approach exploiting hierarchical graph structures and adaptive random walks to address this challenge. Our method introduces learnable transition probabilities that decide whether the walk should prefer the original graph or travel across hierarchical shortcuts. On a synthetic long-range task, we demonstrate that our approach can exceed the theoretical bound that constrains traditional approaches operating solely on the original topology. Specifically, walks that prefer the hierarchy achieve the same performance as longer walks on the original graph. These preliminary findings open a promising direction for efficiently processing large graphs while effectively capturing long-range dependencies.

Problem

• Addressing long-range dependency modeling in graph prediction tasks
• Exploiting hierarchical graph structures with adaptive random walks
• Overcoming theoretical constraints of traditional message-passing architectures

Innovation

• Adaptive random walks on hierarchical graphs
• Learnable transition probabilities for shortcuts
• Exceeding theoretical bounds of traditional approaches