Linearly Convergent Algorithms for Nonsmooth Problems with Unknown Smooth Pieces

2025-07-25

AI Summary

This paper addresses optimization of nonsmooth functions with unknown piecewise structures. For (approximately) piecewise smooth functions satisfying a quadratic growth condition, we propose the first level-set bundle method with guaranteed global linear convergence. Our method integrates a search subroutine with a guess-and-verify framework, enabling adaptive identification of the underlying piecewise structure without prior knowledge or manual parameter tuning. We establish, for the first time, a verifiable, parameter-free exact termination criterion that rigorously quantifies the optimality gap. The algorithm achieves state-of-the-art iteration complexity—matching the best-known rates for smooth nonconvex optimization—under both convex and weakly convex settings. Moreover, it naturally extends to approximately piecewise smooth and weakly convex problems, delivering efficient, robust, and nearly parameter-free optimization.

Technology Category

• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements

Abstract

We develop efficient algorithms for optimizing piecewise smooth (PWS) functions where the underlying partition of the domain into smooth pieces is emph{unknown}. For PWS functions satisfying a quadratic growth (QG) condition, we propose a bundle-level (BL) type method that achieves global linear convergence -- to our knowledge, the first such result for any algorithm for this problem class. We extend this method to handle approximately PWS functions and to solve weakly-convex PWS problems, improving the state-of-the-art complexity to match the benchmark for smooth non-convex optimization. Furthermore, we introduce the first verifiable and accurate termination criterion for PWS optimization. Similar to the gradient norm in smooth optimization, this certificate tightly characterizes the optimality gap under the QG condition, and can moreover be evaluated without knowledge of any problem parameters. We develop a search subroutine for this certificate and embed it within a guess-and-check framework, resulting in an almost parameter-free algorithm for both the convex QG and weakly-convex settings.

Problem

• Optimizing piecewise smooth functions with unknown domain partitions
• Achieving linear convergence for nonsmooth problems with QG condition
• Developing verifiable termination criteria for PWS optimization

Innovation

• Bundle-level method for unknown smooth pieces
• Verifiable termination criterion for PWS optimization
• Parameter-free algorithm for convex QG settings