Reducing Smoothness with Expressive Memory Enhanced Hierarchical Graph Neural Networks

2025-04-01

AI Summary

This paper addresses the excessive representation smoothing and information loss in hierarchical graph neural networks (HGNNs) for multi-scale temporal graph forecasting, caused by unregulated inter-level message passing. To tackle this, we propose Hierarchical Graph Flow (HiGFlow), a novel model featuring a dynamic capacity-constrained memory buffer that explicitly models and regulates hierarchical message flow. We theoretically prove that this mechanism reduces mapping smoothness across levels and enhances Weisfeiler–Lehman graph distinguishability. HiGFlow integrates hierarchical graph structure modeling, dynamic memory augmentation, and optimized message passing. Evaluated on global temporal forecasting tasks—such as weather prediction—HiGFlow achieves average improvements of 6.1% and 6.2% in MAE and RMSE, respectively, over state-of-the-art models including Transformers. It significantly enhances cross-resolution representation quality and forecasting accuracy.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Planning, Routing, and Scheduling: Temporal Planning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Graphical forecasting models learn the structure of time series data via projecting onto a graph, with recent techniques capturing spatial-temporal associations between variables via edge weights. Hierarchical variants offer a distinct advantage by analysing the time series across multiple resolutions, making them particularly effective in tasks like global weather forecasting, where low-resolution variable interactions are significant. A critical challenge in hierarchical models is information loss during forward or backward passes through the hierarchy. We propose the Hierarchical Graph Flow (HiGFlow) network, which introduces a memory buffer variable of dynamic size to store previously seen information across variable resolutions. We theoretically show two key results: HiGFlow reduces smoothness when mapping onto new feature spaces in the hierarchy and non-strictly enhances the utility of message-passing by improving Weisfeiler-Lehman (WL) expressivity. Empirical results demonstrate that HiGFlow outperforms state-of-the-art baselines, including transformer models, by at least an average of 6.1% in MAE and 6.2% in RMSE. Code is available at https://github.com/TB862/ HiGFlow.git.

Problem

• Reduces information loss in hierarchical graph neural networks
• Enhances message-passing utility via improved WL expressivity
• Improves forecasting accuracy over state-of-the-art baselines

Innovation

• Hierarchical Graph Flow with dynamic memory buffer
• Reduces smoothness in feature space mapping
• Enhances message-passing via improved WL expressivity