Exact and efficient basis pursuit denoising via differential inclusions and a selection principle

2025-07-07

AI Summary

Existing Basis Pursuit Denoising (BPDN) algorithms struggle to balance solution accuracy and computational efficiency in high-dimensional settings. This paper proposes a novel dynamical systems approach grounded in differential inclusions and the selection principle. Specifically, we introduce the selection principle to the BPDN dual problem for the first time, constructing an analytically solvable projected dynamical system that enables exact, continuous, and efficient tracking of the regularization path. Building upon this, we derive a strongly polynomial-time greedy algorithm for Basis Pursuit. Our method integrates asymptotic analysis, homotopy continuation, and numerical integration to ensure global convergence and stability. Experiments demonstrate that the proposed algorithm significantly outperforms state-of-the-art methods in reconstruction accuracy, computational speed, and scalability to high dimensions.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Basis pursuit denoising (BPDN) is a cornerstone of compressive sensing, statistics and machine learning. While various algorithms for BPDN have been proposed, they invariably suffer from drawbacks and must either favor efficiency at the expense of accuracy or vice versa. As such, state-of-the-art algorithms remain ineffective for high-dimensional applications that require accurate solutions within a reasonable amount of computational time. In this work, we address this issue and propose an exact and efficient algorithm for BPDN using differential inclusions. Specifically, we prove that a selection principle from the theory of differential inclusions turns the dual problem of BPDN into calculating the trajectory of an emph{integrable} projected dynamical system, that is, whose trajectory and asymptotic limit can be computed exactly. Our analysis naturally yields an exact algorithm, numerically up to machine precision, that is amenable to computing regularization paths and very fast. Numerical experiments confirm that our algorithm outperforms the state-of-the-art algorithms in both accuracy and efficiency. Moreover, we show that the global continuation of solutions (in terms of the hyperparameter and data) of the projected dynamical system yields a rigorous homotopy algorithm for BPDN, as well as a novel greedy algorithm for computing feasible solutions to basis pursuit in strongly polynomial time. Beyond this work, we expect that our results and analysis can be adapted to compute exact or approximate solutions to a broader class of polyhedral-constrained optimization problems.

Problem

• Exact and efficient algorithm for basis pursuit denoising (BPDN)
• Overcoming trade-offs between accuracy and efficiency in BPDN
• High-dimensional applications requiring precise solutions quickly

Innovation

• Uses differential inclusions for exact BPDN solutions
• Integrable projected dynamical system for efficiency
• Global continuation enables rigorous homotopy algorithm