Fast and Accurate Algorithms to Calculate Expected Modularity in Probabilistic Networks

2024-08-13

arXiv.org

AI Summary

Computing the exact expected modularity in probabilistic networks is computationally intractable and inefficient due to exponential enumeration of possible worlds. To address this, we propose PWP^EMOD and APWP^EMOD, the first scalable algorithms for exact expected modularity computation. Our approach innovatively groups possible worlds by modular structure—leveraging the modularity distribution—thereby avoiding exhaustive enumeration and drastically reducing the number of probability summations. The algorithms integrate probabilistic graphical models, dynamic programming, and combinatorial optimization techniques. Extensive evaluation on diverse real-world and synthetic datasets demonstrates that our methods achieve up to three orders-of-magnitude speedup over state-of-the-art baselines, with absolute error under 0.5%. They scale to networks with tens of thousands of edges and enable real-time probabilistic community assessment—overcoming the long-standing scalability bottleneck in probabilistic community detection evaluation.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Probabilistic Circuits and Graphical Models
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Community question answering
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Modularity maximization is a widely used community detection technique for deterministic networks. However, little research has been performed to develop efficient modularity calculation algorithms for probabilistic networks. Particularly, it is challenging to efficiently calculate expected modularity when all possible worlds are considered. To address this problem, we propose two algorithms, namely $mathrm{PWP}^{mathrm{EMOD}}$ and $mathrm{APWP}^{mathrm{EMOD}}$, partitioning the possible worlds based on their modularities to significantly reduce the number of probability calculations. We evaluate the accuracy and time efficiency of our algorithms through comprehensive experiments.

Problem

• Efficiently compute expected modularity in probabilistic networks
• Compare accuracy of methods for modularity computation
• Propose fast, accurate technique (FPWP) for modularity distribution

Innovation

• Novel FPWP technique for modularity computation
• Accurate probability distribution of modularity
• Efficient expected value calculation in networks