Perfect reconstruction of sparse signals using nonconvexity control and one-step RSB message passing

2025-12-19

AI Summary

This paper addresses the problem of perfect reconstruction of sparse signals. We propose a first-order replica-symmetric-breaking approximate message-passing algorithm (1RSB-AMP) incorporating the smoothly clipped absolute deviation (SCAD) nonconvex penalty—the first integration of 1RSB variational inference into the AMP framework. Our key contributions include: (i) a novel Parisi parameter selection criterion that minimizes the divergence region, relaxing the conventional zero-complexity constraint; and (ii) incorporation of a nonconvexity control (NCC) protocol to ensure algorithmic convergence. Theoretical analysis employs 1RSB state evolution (1RSB-SE) to characterize phase-transition behavior. Numerical experiments demonstrate that our algorithm achieves a significantly improved reconstruction threshold compared to RS-AMP, approaching the Bayes-optimal limit; exhibits robust convergence outside the divergence region; and yields a theoretical phase diagram in excellent agreement with simulations.

Technology Category

• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

We consider sparse signal reconstruction via minimization of the smoothly clipped absolute deviation (SCAD) penalty, and develop one-step replica-symmetry-breaking (1RSB) extensions of approximate message passing (AMP), termed 1RSB-AMP. Starting from the 1RSB formulation of belief propagation, we derive explicit update rules of 1RSB-AMP together with the corresponding state evolution (1RSB-SE) equations. A detailed comparison shows that 1RSB-AMP and 1RSB-SE agree remarkably well at the macroscopic level, even in parameter regions where replica-symmetric (RS) AMP, termed RS-AMP, diverges and where the 1RSB description itself is not expected to be thermodynamically exact. Fixed-point analysis of 1RSB-SE reveals a phase diagram consisting of success, failure, and diverging phases, as in the RS case. However, the diverging-region boundary now depends on the Parisi parameter due to the 1RSB ansatz, and we propose a new criterion -- minimizing the size of the diverging region -- rather than the conventional zero-complexity condition, to determine its value. Combining this criterion with the nonconvexity-control (NCC) protocol proposed in a previous RS study improves the algorithmic limit of perfect reconstruction compared with RS-AMP. Numerical solutions of 1RSB-SE and experiments with 1RSB-AMP confirm that this improved limit is achieved in practice, though the gain is modest and remains slightly inferior to the Bayes-optimal threshold. We also report the behavior of thermodynamic quantities -- overlaps, free entropy, complexity, and the non-self-averaging susceptibility -- that characterize the 1RSB phase in this problem.

Problem

• Develops 1RSB-AMP algorithm for sparse signal reconstruction using SCAD penalty.
• Analyzes phase diagram and proposes criterion to minimize diverging region.
• Improves perfect reconstruction limit via nonconvexity control and 1RSB extensions.

Innovation

• Developed 1RSB-AMP algorithm for sparse signal reconstruction
• Used nonconvexity control protocol to improve reconstruction limit
• Proposed new criterion minimizing diverging region size