Probabilistic Gaussian Homotopy: A Probability-Space Continuation Framework for Nonconvex Optimization

2026-03-13

AI Summary

This work proposes a probabilistic space continuation framework to address the challenges of non-convex optimization, particularly the susceptibility to local minima and the failure of conventional gradient-based and objective-space smoothing techniques. The approach leverages a deformed Boltzmann distribution to weight and aggregate perturbed gradients, thereby steering the search trajectory toward low-energy regions. It establishes a theoretical connection among Gaussian homotopy, Bayesian denoising, and diffusion-based smoothing, generalizing posterior-mean-type Moreau envelopes through a log-sum-exp soft-minimization formulation. By integrating annealed dynamical systems modeling with Monte Carlo gradient estimation, the method demonstrates significant performance improvements over existing approaches on high-dimensional non-convex benchmarks and sparse recovery tasks.

Technology Category

• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Learning with Manifolds

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Content-based information diffusion
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

We introduce Probabilistic Gaussian Homotopy (PGH), a probability-space continuation framework for nonconvex optimization. Unlike classical Gaussian homotopy, which smooths the objective and uniformly averages gradients, PGH deforms the associated Boltzmann distribution and induces Boltzmann-weighted aggregation of perturbed gradients, which exponentially biases descent directions toward low-energy regions. We show that PGH corresponds to a log-sum-exp (soft-min) homotopy that smooths a nonconvex objective at scale $λ>0$ and recovers the original objective as $λ\to 0$, yielding a posterior-mean generalization of the Moreau envelope, and we derive a dynamical system governing minimizer evolution along an annealed homotopy path. This establishes a principled connection between Gaussian continuation, Bayesian denoising, and diffusion-style smoothing. We further propose Probabilistic Gaussian Homotopy Optimization (PGHO), a practical stochastic algorithm based on Monte Carlo gradient estimation, and demonstrate strong performance on high-dimensional nonconvex benchmarks and sparse recovery problems where classical gradient methods and objective-space smoothing frequently fail.

Problem

• nonconvex optimization
• Gaussian homotopy
• Boltzmann distribution
• sparse recovery
• high-dimensional optimization

Innovation

• Probabilistic Gaussian Homotopy
• nonconvex optimization
• Boltzmann-weighted aggregation
• log-sum-exp homotopy
• Monte Carlo gradient estimation