The Ball-Proximal (="Broximal") Point Method: a New Algorithm, Convergence Theory, and Applications

2025-02-04

AI Summary

Addressing the classical challenge of nonsmooth nonconvex global optimization—ubiquitous in machine learning, control, and signal processing—this paper proposes the Ball Proximal Method (BPM), which replaces the quadratic penalty term in standard proximal operators with a spherical constraint to establish a novel optimization framework. We introduce the notion of “ball convexity,” enabling linear and finite-step convergence for nonsmooth convex problems—surpassing the sublinear rate limitation of classical proximal methods. Under weaker assumptions, we further establish global convergence guarantees for the nonconvex case. Theoretically, BPM achieves both linear convergence rates and finite termination. Moreover, our analysis unifies key algorithmic principles—including acceleration mechanisms, adaptive stepsize selection, smoothing techniques, and trust-region ideas—revealing their intrinsic connections. This work provides a new paradigm for designing efficient algorithms for nonsmooth nonconvex optimization.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Non-smooth and non-convex global optimization poses significant challenges across various applications, where standard gradient-based methods often struggle. We propose the Ball-Proximal Point Method, Broximal Point Method, or Ball Point Method (BPM) for short - a novel algorithmic framework inspired by the classical Proximal Point Method (PPM) (Rockafellar, 1976), which, as we show, sheds new light on several foundational optimization paradigms and phenomena, including non-convex and non-smooth optimization, acceleration, smoothing, adaptive stepsize selection, and trust-region methods. At the core of BPM lies the ball-proximal ("broximal") operator, which arises from the classical proximal operator by replacing the quadratic distance penalty by a ball constraint. Surprisingly, and in sharp contrast with the sublinear rate of PPM in the nonsmooth convex regime, we prove that BPM converges linearly and in a finite number of steps in the same regime. Furthermore, by introducing the concept of ball-convexity, we prove that BPM retains the same global convergence guarantees under weaker assumptions, making it a powerful tool for a broader class of potentially non-convex optimization problems. Just like PPM plays the role of a conceptual method inspiring the development of practically efficient algorithms and algorithmic elements, e.g., gradient descent, adaptive step sizes, acceleration (Ahn&Sra, 2020), and"W"in AdamW (Zhuang et al., 2022), we believe that BPM should be understood in the same manner: as a blueprint and inspiration for further development.

Problem

• Develops Ball-Proximal Point Method for non-convex optimization
• Proves linear convergence in non-smooth convex optimization
• Introduces ball-convexity for broader global convergence guarantees

Innovation

• Ball-Proximal Point Method
• Linear and finite-step convergence
• Ball-convexity for global guarantees