Learning CNN Filters via Generalized Stein's Method

2026-09-02

AI Summary

本文通过广义Stein方法从统计角度估计卷积滤波器，提出基于奇异值分解的新方法，实现滤波器的准确学习。

Abstract

Convolutional Neural Networks (CNNs) have undoubtedly revolutionized image data analysis and the field of computer vision. As the cornerstone of CNNs, the convolution operation enables the networks to extract abstract features and uncover hidden relationships in the image data. This paper considers the problem of estimating convolution filters from a statistical perspective using a classical tool --- Stein's formula. We first formulate CNNs into a general index model with matrix-valued input, where convolution filters can be viewed as index vectors. Furthermore, we propose a novel singular value decomposition (SVD) based approach to accurately learn the convolution filters based on a generalized version of the first-order Stein's formula. Theoretical analysis suggests that our estimation achieves an optimal convergence rate, comparable to that of generalized linear models where the link function is known. Extensive simulation studies and real data analyses demonstrate that our approach outperforms popular deep learning algorithms, such as Adam. Notably, our method extends beyond filter estimation and can be applied to nonlinear dimension reduction, providing a viable pathway for representation learning.

Problem

• Convolutional Neural Networks
• Stein's formula
• filter estimation

Innovation

• Generalized Stein's Method
• Singular Value Decomposition (SVD)
• Optimal Convergence Rate
• Nonlinear Dimension Reduction