Discovering Sparse Recovery Algorithms Using Neural Architecture Search

2025-12-25

AI Summary

Conventional inverse problem algorithms in signal processing rely heavily on hand-crafted designs, suffering from poor generalizability and limited adaptability across diverse data distributions and algorithmic families. Method: This paper introduces neural architecture search (NAS) to automated sparse recovery algorithm discovery—the first such application. We propose a differentiable meta-learning framework that jointly optimizes the iterative update structure, soft-thresholding operator, and acceleration mechanism over a search space of >50,000 candidate operations, effectively reconstructing the core update logic of ISTA/FISTA. Contribution/Results: Our method precisely reproduces the convergence behavior and acceleration properties of classical algorithms while demonstrating strong cross-distribution and cross-algorithm-family generalization. It breaks away from heuristic design paradigms, establishing the first scalable and interpretable automated framework for inverse problem algorithm discovery—enabling principled, data-aware, and architecture-agnostic solver design.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Optimization
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

The design of novel algorithms for solving inverse problems in signal processing is an incredibly difficult, heuristic-driven, and time-consuming task. In this short paper, we the idea of automated algorithm discovery in the signal processing context through meta-learning tools such as Neural Architecture Search (NAS). Specifically, we examine the Iterative Shrinkage Thresholding Algorithm (ISTA) and its accelerated Fast ISTA (FISTA) variant as candidates for algorithm rediscovery. We develop a meta-learning framework which is capable of rediscovering (several key elements of) the two aforementioned algorithms when given a search space of over 50,000 variables. We then show how our framework can apply to various data distributions and algorithms besides ISTA/FISTA.

Problem

• Automated discovery of sparse recovery algorithms via neural architecture search.
• Rediscovering ISTA and FISTA algorithms from a large search space.
• Extending the framework to diverse data distributions and algorithms.

Innovation

• Neural Architecture Search automates algorithm discovery
• Meta-learning framework rediscovered ISTA and FISTA algorithms
• Framework handles over 50,000 variables and diverse data distributions