Can You Hear Me Now? A Benchmark for Long-Range Graph Propagation

2025-12-19

AI Summary

Graph Neural Networks (GNNs) struggle to effectively model long-range interactions (>10 hops), limiting their applicability in molecular science and related domains. To address this, we propose ECHO—a dedicated, structured benchmark for evaluating long-range information propagation. ECHO comprises three synthetic graph tasks with controllable topologies (shortest path, eccentricity, diameter) and two density functional theory (DFT)-level real-world molecular property prediction tasks (atomic charge distribution and total energy). It introduces structured information-bottleneck graphs and chemically interpretable tasks to rigorously assess mainstream GNNs—including GCN, GAT, GIN, and PNA. Experimental results reveal a 40–70% performance degradation on long-range tasks across all models; message aggregation mechanisms and skip-connection designs emerge as primary bottlenecks. ECHO provides a reproducible diagnostic tool and a new evaluation standard for assessing long-range reasoning capabilities in AI for Science.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Effectively capturing long-range interactions remains a fundamental yet unresolved challenge in graph neural network (GNN) research, critical for applications across diverse fields of science. To systematically address this, we introduce ECHO (Evaluating Communication over long HOps), a novel benchmark specifically designed to rigorously assess the capabilities of GNNs in handling very long-range graph propagation. ECHO includes three synthetic graph tasks, namely single-source shortest paths, node eccentricity, and graph diameter, each constructed over diverse and structurally challenging topologies intentionally designed to introduce significant information bottlenecks. ECHO also includes two real-world datasets, ECHO-Charge and ECHO-Energy, which define chemically grounded benchmarks for predicting atomic partial charges and molecular total energies, respectively, with reference computations obtained at the density functional theory (DFT) level. Both tasks inherently depend on capturing complex long-range molecular interactions. Our extensive benchmarking of popular GNN architectures reveals clear performance gaps, emphasizing the difficulty of true long-range propagation and highlighting design choices capable of overcoming inherent limitations. ECHO thereby sets a new standard for evaluating long-range information propagation, also providing a compelling example for its need in AI for science.

Problem

• Evaluating GNNs' ability to handle long-range graph propagation
• Introducing a benchmark with synthetic and real-world molecular tasks
• Identifying performance gaps in current GNN architectures for long-range interactions

Innovation

• Introduces ECHO benchmark for long-range GNN evaluation
• Includes synthetic tasks and real-world molecular datasets
• Reveals performance gaps in existing GNN architectures