Simultaneous estimation of connectivity and dimensionality in samples of networks

2025-08-17

AI Summary

Estimating the low-rank connection probability matrix and selecting its rank (embedding dimension) are inherently coupled challenges in multi-network data analysis. Method: This paper proposes a joint optimization framework that simultaneously estimates both the low-rank probability matrix and its rank, given a pre-estimated node community structure (i.e., number of communities and membership assignments). The approach formulates a convex optimization problem integrating nuclear norm regularization with stochastic block model theory, solved efficiently via the Alternating Direction Method of Multipliers (ADMM). Theoretical analysis establishes statistical consistency even under imperfect community label estimation. Contribution/Results: Compared to conventional network averaging, our method achieves significantly improved accuracy in estimating both the connection structure and the true rank. Extensive simulations confirm its robustness and estimation accuracy. Applied to primate brain connectome data, it empirically validates that the underlying connectivity matrix exhibits strong low-rank structure—i.e., is markedly non-full-rank.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Social Networks and Social Media: Computational social science
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

An overarching objective in contemporary statistical network analysis is extracting salient information from datasets consisting of multiple networks. To date, considerable attention has been devoted to node and network clustering, while comparatively less attention has been devoted to downstream connectivity estimation and parsimonious embedding dimension selection. Given a sample of potentially heterogeneous networks, this paper proposes a method to simultaneously estimate a latent matrix of connectivity probabilities and its embedding dimensionality or rank after first pre-estimating the number of communities and the node community memberships. The method is formulated as a convex optimization problem and solved using an alternating direction method of multipliers algorithm. We establish estimation error bounds under the Frobenius norm and nuclear norm for settings in which observable networks have blockmodel structure, even when node memberships are imperfectly recovered. When perfect membership recovery is possible and dimensionality is much smaller than the number of communities, the proposed method outperforms conventional averaging-based methods for estimating connectivity and dimensionality. Numerical studies empirically demonstrate the accuracy of our method across various scenarios. Additionally, analysis of a primate brain dataset demonstrates that posited connectivity is not necessarily full rank in practice, illustrating the need for flexible methodology.

Problem

• Estimating connectivity and dimensionality in multiple heterogeneous networks
• Developing a method for simultaneous connectivity and rank estimation
• Improving accuracy over conventional averaging-based estimation methods

Innovation

• Simultaneous estimation of connectivity and dimensionality
• Convex optimization with ADMM algorithm
• Error bounds under Frobenius and nuclear norms