Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation

2024-02-05

Neural Information Processing Systems

AI Summary

This work addresses the non-asymptotic convergence of SGD with biased gradients and adaptive step sizes in nonconvex smooth optimization. We establish, for the first time, non-asymptotic convergence guarantees for AdaGrad, RMSProp, and AMSGrad under gradient bias. Our analysis shows that if the bias decays at rate $O(1/sqrt{t})$, all three algorithms converge to a critical point at rate $O(1/sqrt{T})$; crucially, the controllability of bias—not merely its presence—governs the convergence rate. Methodologically, we integrate non-asymptotic analysis, Monte Carlo modeling of stochastic gradient estimation, and adaptive optimization theory. Empirical validation on generative models—including VAEs—demonstrates that careful hyperparameter tuning (e.g., learning rate scaling and bias-variance trade-offs) substantially mitigates bias-induced instability and improves both training robustness and final performance.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Bias, Fairness & Privacy

Application Category

• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Stochastic Gradient Descent (SGD) with adaptive steps is widely used to train deep neural networks and generative models. Most theoretical results assume that it is possible to obtain unbiased gradient estimators, which is not the case in several recent deep learning and reinforcement learning applications that use Monte Carlo methods. This paper provides a comprehensive non-asymptotic analysis of SGD with biased gradients and adaptive steps for non-convex smooth functions. Our study incorporates time-dependent bias and emphasizes the importance of controlling the bias of the gradient estimator. In particular, we establish that Adagrad, RMSProp, and AMSGRAD, an exponential moving average variant of Adam, with biased gradients, converge to critical points for smooth non-convex functions at a rate similar to existing results in the literature for the unbiased case. Finally, we provide experimental results using Variational Autoenconders (VAE) and applications to several learning frameworks that illustrate our convergence results and show how the effect of bias can be reduced by appropriate hyperparameter tuning.

Problem

• Analyzes SGD with biased gradients and adaptive steps.
• Focuses on non-convex smooth functions and time-dependent bias.
• Demonstrates convergence of Adagrad, RMSProp, and AMSGRAD with bias.

Innovation

• Non-asymptotic analysis of biased SGD
• Convergence of adaptive methods with bias
• Hyperparameter tuning reduces bias effects