Towards a Sampling Theory for Implicit Neural Representations

2024-05-28

Asilomar Conference on Signals, Systems and Computers

AI Summary

This work investigates the sample complexity of implicit neural representations (INRs) for linear inverse problems—specifically, continuous-domain image reconstruction from low-frequency Fourier coefficients. Methodologically, it integrates Fourier feature mapping with generalized weight decay regularization, and employs variational analysis in the space of Radon measures to characterize reconstruction boundaries. Theoretically, it establishes, for the first time, an equivalence between training single-hidden-layer ReLU INRs and convex regularization over measures, yielding a sufficient sampling condition for exact recovery when the width is one, and proposing a verifiable critical sampling conjecture for arbitrary widths. Experimentally, it demonstrates that low-width INRs achieve high-fidelity recovery of continuous-domain aliasing artifacts in super-resolution, revealing a fundamental link between their low-frequency inductive bias and sampling efficiency.

Technology Category

• Machine Learning: Representation Learning
• Computer Vision: Representation Learning for Vision
• Search and Optimization: Non-convex Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Implicit neural representations (INRs) have emerged as a powerful tool for solving inverse problems in computer vision and computational imaging. INRs represent images as continuous domain functions realized by a neural network taking spatial coordinates as inputs. However, unlike traditional pixel representations, little is known about the sample complexity of estimating images using INRs in the context of linear inverse problems. Towards this end, we study the sampling requirements for recovery of a continuous domain image from its low-pass Fourier coefficients by fitting a single hidden-layer INR with ReLU activation and a Fourier features layer using a generalized form of weight decay regularization. Our key insight is to relate minimizers of this non-convex parameter space optimization problem to minimizers of a convex penalty defined over a space of measures. We identify a sufficient number of samples for which an image realized by a width-1 INR is exactly recoverable by solving the INR training problem, and give a conjecture for the general width $W$ case. To validate our theory, we empirically assess the probability of achieving exact recovery of images realized by low-width single hidden-layer INRs, and illustrate the performance of INR on super-resolution recovery of more realistic continuous domain phantom images.

Problem

• Study sample complexity for image recovery using INRs
• Determine sufficient samples for exact INR-based recovery
• Validate theory with empirical INR recovery experiments

Innovation

• Single hidden-layer INR with ReLU activation
• Fourier features layer for image recovery
• Generalized weight decay regularization technique