Sparse Network Inference under Imperfect Detection and its Application to Ecological Networks

2026-04-20

AI Summary

This study addresses the challenge of simultaneously recovering both cross-group interactions and within-group similarity structures in sparse ecological networks from count data corrupted by detection errors. To this end, the authors propose a structured sparse nonnegative low-rank factorization framework that incorporates detection probability estimation. The method jointly models within-group similarity graphs and cross-group connectivity through non-convex ℓ₁/₂ regularization and employs an ADMM-based optimization algorithm enhanced with adaptive penalization and scale-aware initialization. Notably, it achieves the first end-to-end joint optimization of detection probabilities, similarity graphs, and interaction structures, accompanied by theoretical convergence guarantees. Experimental results on both synthetic and real ecological datasets demonstrate that the proposed approach significantly outperforms existing baselines in accurately recovering latent factors and underlying network topologies.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Probabilistic Circuits and Graphical Models
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Recovering latent structure from count data has received considerable attention in network inference, particularly when one seeks both cross-group interactions and within-group similarity patterns in bipartite networks, which is widely used in ecology research. Such networks are often sparse and inherently imperfect in their detection. Existing models mainly focus on interaction recovery, while the induced similarity graphs are much less studied. Moreover, sparsity is often not controlled, and scale is unbalanced, leading to oversparse or poorly rescaled estimates with degrading structural recovery. To address these issues, we propose a framework for structured sparse nonnegative low-rank factorization with detection probability estimation. We impose nonconvex $\ell_{1/2}$ regularization on the latent similarity and connectivity structures to promote sparsity within-group similarity and cross-group connectivity with better relative scale. The resulting optimization problem is nonconvex and nonsmooth. To solve it, we develop an ADMM-based algorithm with adaptive penalization and scale-aware initialization and establish its asymptotic feasibility and KKT stationarity of cluster points under mild regularity conditions. Experiments on synthetic and real-world ecological datasets demonstrate improved recovery of latent factors and similarity/connectivity structure relative to existing baselines.

Problem

• Sparse Network Inference
• Imperfect Detection
• Ecological Networks
• Latent Structure Recovery
• Similarity Graph

Innovation

• structured sparsity
• nonconvex regularization
• low-rank factorization
• imperfect detection
• ADMM algorithm