The Performance Of The Unadjusted Langevin Algorithm Without Smoothness Assumptions

2025-02-05

AI Summary

This work addresses efficient sampling from nonsmooth and non-log-concave target distributions. We propose a Langevin algorithm that operates directly on the original (potentially nondifferentiable) density—requiring neither smoothing preprocessing nor hyperparameter tuning. For the first time, we establish a non-asymptotic Wasserstein-2 convergence theory for the Unadjusted Langevin Algorithm (ULA) under *no* smoothness or log-concavity assumptions, bypassing computationally expensive techniques such as the Moreau–Yosida envelope or Gaussian smoothing. The algorithm achieves an $O(1/sqrt{k})$ convergence rate in Wasserstein-2 distance after $k$ iterations. We further derive an explicit generalization error bound for excess risk minimization. Our theoretical framework accommodates ill-conditioned and irregular target distributions, substantially broadening both the applicability and practical utility of Langevin-based sampling methods.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Learning with Manifolds
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

In this article, we study the problem of sampling from distributions whose densities are not necessarily smooth nor log-concave. We propose a simple Langevin-based algorithm that does not rely on popular but computationally challenging techniques, such as the Moreau Yosida envelope or Gaussian smoothing. We derive non-asymptotic guarantees for the convergence of the algorithm to the target distribution in Wasserstein distances. Non asymptotic bounds are also provided for the performance of the algorithm as an optimizer, specifically for the solution of associated excess risk optimization problems.

Problem

• Sampling from non-smooth, non-log-concave distributions efficiently
• Proposing a simple Langevin algorithm without complex techniques
• Providing non-asymptotic convergence guarantees in Wasserstein distances

Innovation

• Langevin-based algorithm without smoothness assumptions
• Convergence guarantees in Wasserstein distances
• Non-asymptotic bounds for excess risk optimization