Efficient Learning on Large Graphs using a Densifying Regularity Lemma

2025-04-25

AI Summary

To address the scalability bottleneck of Graph Neural Networks (GNNs) in large-scale graph learning—where computation and memory costs scale linearly with the number of edges, $O(m)$—this paper proposes the Intersecting Block Graph (IBG), a low-rank graph approximation framework grounded in overlapping bipartite community structure. We provide the first constructive proof of the weak regularity lemma, establishing that the approximation rank depends only on accuracy, not sparsity. This enables node-level $O(n)$ time and space complexity, breaking the edge-dependent constraint of conventional GNNs. IBG supports efficient, compositional low-rank decomposition over communities and integrates seamlessly into standard GNN architectures. Empirically, IBG achieves state-of-the-art performance on node classification, spatiotemporal graph forecasting, and knowledge graph completion, significantly improving both scalability and predictive accuracy.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Learning on large graphs presents significant challenges, with traditional Message Passing Neural Networks suffering from computational and memory costs scaling linearly with the number of edges. We introduce the Intersecting Block Graph (IBG), a low-rank factorization of large directed graphs based on combinations of intersecting bipartite components, each consisting of a pair of communities, for source and target nodes. By giving less weight to non-edges, we show how to efficiently approximate any graph, sparse or dense, by a dense IBG. Specifically, we prove a constructive version of the weak regularity lemma, showing that for any chosen accuracy, every graph, regardless of its size or sparsity, can be approximated by a dense IBG whose rank depends only on the accuracy. This dependence of the rank solely on the accuracy, and not on the sparsity level, is in contrast to previous forms of the weak regularity lemma. We present a graph neural network architecture operating on the IBG representation of the graph and demonstrating competitive performance on node classification, spatio-temporal graph analysis, and knowledge graph completion, while having memory and computational complexity linear in the number of nodes rather than edges.

Problem

• Reducing computational and memory costs in large graph learning
• Approximating any graph efficiently using dense Intersecting Block Graphs
• Enabling linear complexity in node classification and graph analysis

Innovation

• Low-rank graph factorization using Intersecting Block Graph
• Dense approximation of graphs with accuracy-dependent rank
• Graph neural network with linear node-based complexity