Compressive Sensing - Introduction and Relations to Deep Learning

2026-08-25

AI Summary

本文介绍了压缩感知技术如何通过少量线性测量恢复稀疏信号，并探讨了其与深度学习的联系，特别是展开神经网络和梯度下降法在稀疏恢复中的应用。

Abstract

Compressive sensing predicts that sparse vectors (signals) can be recovered from a small number of linear measurements via efficient algorithms. This finding, which dates back two decades, has triggered a paradigm shift in signal processing and initiated many developments both in practical signal-processing applications, such as medical imaging, radar, and astronomy, and on the theoretical side. More recently, seminal connections to the field of deep learning have led to further advances in the field, such as the use of unrolled neural networks for sparse recovery and the discovery that common training algorithms (variants of gradient descent) favor sparsity in overparameterized scenarios -- the so-called implicit bias phenomenon. This article gives an introduction to compressive sensing and outlines connections to deep learning. In particular, we will discuss generalization for neural networks generated by unrolling sparse recovery algorithms and implicit regularization for gradient descent applied to learning simplified linear neural networks.

Problem

• compressive sensing
• deep learning
• sparse recovery
• generalization
• implicit regularization

Innovation

• Compressive Sensing
• Deep Learning
• Unrolled Neural Networks
• Sparse Recovery
• Implicit Regularization