Memory AMP: Overflow Avoidance, Complexity Reduction, and Comparative Analysis

2026-08-09

AI Summary

This work addresses the numerical instability and high computational complexity of gradient descent-based memory approximate message passing (GD-MAMP) under ill-conditioned settings, where intermediate variable overflow occurs. To mitigate these issues, the authors propose a low-complexity variant that incorporates a partial memory mechanism and reduces the number of matrix-vector multiplications per iteration by one-third to one-half. A unified gradient framework is developed, revealing that the instability of WS-CG-VAMP in finite precision stems from catastrophic cancellation, and an efficient reformulation, termed WS-CG-VAMP(r), is derived. The proposed methods significantly enhance numerical stability and computational efficiency while preserving convergence speed: GD-MAMP remains suitable for well-conditioned problems, whereas WS-CG-VAMP(r) excels in high-precision scenarios with large condition numbers.

Abstract

Approximate message passing (AMP)-type algorithms are widely used for signal recovery in high-dimensional noisy linear systems. Recently, a framework called memory AMP (MAMP) was introduced, offering a new approach to incorporating memory terms within AMP algorithms. Building on this, a low-complexity gradient descent MAMP (GD-MAMP) was proposed for right-unitarily invariant matrices. In this paper, we first address an overflow problem in GD-MAMP caused by intermediate variables exceeding the floating-point range, which typically occurs when the condition number is large. Second, we propose two low-complexity variants of GD-MAMP: one replaces full-length memory with partial memory, while the other reduces the number of matrix-vector products per iteration by $1/3$ (from three to two). Neither degrades the convergence speed notably. Third, we develop a general gradient-based formulation for designing MAMP algorithms. This formulation recovers warm-started conjugate gradient VAMP (WS-CG-VAMP) as a special case. Furthermore, we show that the computation of the orthogonalization parameters in this formulation can suffer from catastrophic cancellation, which explains the finite-precision instability of WS-CG-VAMP. Finally, we derive an equivalent reformulation, termed WS-CG-VAMP(r), which reduces the number of matrix-vector products by up to $50\%$. Measured by matrix-vector products, GD-MAMP converges faster for small condition numbers, whereas WS-CG-VAMP(r) converges faster for large ones under high-precision arithmetic but may diverge in IEEE double precision due to catastrophic cancellation.

Problem

• overflow
• complexity
• catastrophic cancellation
• signal recovery
• numerical instability

Innovation

• Memory AMP
• Gradient Descent
• Complexity Reduction
• Catastrophic Cancellation
• Matrix-Vector Products