Convergence of gradient flow for learning convolutional neural networks

2026-01-13

AI Summary

This work addresses the long-standing lack of theoretical guarantees for the convergence of training in convolutional neural networks within the non-convex optimization setting. Focusing on linear convolutional neural networks, the study analyzes the gradient flow dynamics under the empirical risk minimization framework and establishes, for the first time in this setting, a rigorous global convergence guarantee. Under mild assumptions on the data and with squared loss, the authors prove that the gradient flow always converges to a critical point. This result overcomes a key obstacle in the theoretical analysis of non-convex optimization and offers a novel perspective for understanding the optimization behavior of deep convolutional models.

Technology Category

• Search and Optimization: Non-convex Optimization
• Computer Vision: Learning & Optimization for CV
• Machine Learning: Deep Learning Theory

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Security and Privacy: Large-scale security measurements

Abstract

Convolutional neural networks are widely used in imaging and image recognition. Learning such networks from training data leads to the minimization of a non-convex function. This makes the analysis of standard optimization methods such as variants of (stochastic) gradient descent challenging. In this article we study the simplified setting of linear convolutional networks. We show that the gradient flow (to be interpreted as an abstraction of gradient descent) applied to the empirical risk defined via certain loss functions including the square loss always converges to a critical point, under a mild condition on the training data.

Problem

• convolutional neural networks
• gradient flow
• non-convex optimization
• convergence
• empirical risk

Innovation

• gradient flow
• convolutional neural networks
• convergence analysis
• non-convex optimization
• linear networks