Sampling nodes and hyperedges via random walks on large hypergraphs

2025-02-26

AI Summary

Structural representation in large-scale hypergraphs is hindered by high computational complexity and the impossibility of accessing full data. Method: We propose a non-backtracking higher-order random walk that jointly samples nodes and hyperedges—marking the first integration of non-backtracking mechanisms into hypergraph random walks. This approach establishes an unbiased estimation framework, overcoming traditional limitations of local neighborhood access and enabling global structural inference from sparse samples. Leveraging hypergraph adjacency tensor modeling and visit-frequency statistics, our method estimates structural properties directly from sampled trajectories. Contribution/Results: Evaluated on the OpenAlex collaboration hypergraph, our method achieves high-fidelity reconstruction of key metrics—including author productivity and team size—with less than 1% sampling rate and sub-5% estimation error. It significantly outperforms existing baselines, demonstrating superior effectiveness and robustness in real-world complex systems.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Web Mining and Content Analysis: Bridging structured and unstructured data

Abstract

Hypergraphs provide a fundamental framework for representing complex systems involving interactions among three or more entities. As empirical hypergraphs grow in size, characterizing their structural properties becomes increasingly challenging due to computational complexity and, in some cases, restricted access to complete data, requiring efficient sampling methods. Random walks offer a practical approach to hypergraph sampling, as they rely solely on local neighborhood information from nodes and hyperedges. In this study, we investigate methods for simultaneously sampling nodes and hyperedges via random walks on large hypergraphs. First, we compare three existing random walks in the context of hypergraph sampling and identify an advantage of the so-called higher-order random walk. Second, by extending an established technique for graphs to the case of hypergraphs, we present a non-backtracking variant of the higher-order random walk. We derive theoretical results on estimators based on the non-backtracking higher-order random walk and validate them through numerical simulations on large empirical hypergraphs. Third, we apply the non-backtracking higher-order random walk to a large hypergraph of co-authorships indexed in the OpenAlex database, where full access to the data is not readily available. Despite the relatively small sample size, our estimates largely align with previous findings on author productivity, team size, and the prevalence of open-access publications. Our findings contribute to the development of analysis methods for large hypergraphs, offering insights into sampling strategies and estimation techniques applicable to real-world complex systems.

Problem

• Sampling nodes and hyperedges efficiently
• Characterizing large hypergraphs' structural properties
• Extending random walk techniques to hypergraphs

Innovation

• Random walks sample hypergraphs efficiently
• Higher-order random walk enhances sampling
• Non-backtracking variant improves estimation accuracy