Revisiting Frank-Wolfe for Structured Nonconvex Optimization

2025-03-11

AI Summary

This paper studies structured nonconvex optimization problems where the objective function admits a difference-of-convex (DC) decomposition. Addressing the limitation of existing Frank–Wolfe methods—which apply only to convex or smooth nonconvex settings—we propose the first Frank–Wolfe-style algorithm for general DC nonconvex optimization. Our method is projection-free and incorporates a structure-aware DC decomposition that enables geometrically adaptive updates. We establish theoretical guarantees: the algorithm converges to a first-order stationary point with an iteration complexity of $O(1/varepsilon^2)$; under a suitable DC decomposition, the gradient oracle complexity improves to $O(1/varepsilon)$. Empirical evaluations demonstrate that our approach significantly outperforms the standard Frank–Wolfe algorithm in both solution quality and computational efficiency across diverse structured nonconvex benchmarks.

Technology Category

• Search and Optimization: Non-convex Optimization
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Web Mining and Content Analysis: Bridging structured and unstructured data

Abstract

We introduce a new projection-free (Frank-Wolfe) method for optimizing structured nonconvex functions that are expressed as a difference of two convex functions. This problem class subsumes smooth nonconvex minimization, positioning our method as a promising alternative to the classical Frank-Wolfe algorithm. DC decompositions are not unique; by carefully selecting a decomposition, we can better exploit the problem structure, improve computational efficiency, and adapt to the underlying problem geometry to find better local solutions. We prove that the proposed method achieves a first-order stationary point in $O(1/epsilon^2)$ iterations, matching the complexity of the standard Frank-Wolfe algorithm for smooth nonconvex minimization in general. Specific decompositions can, for instance, yield a gradient-efficient variant that requires only $O(1/epsilon)$ calls to the gradient oracle. Finally, we present numerical experiments demonstrating the effectiveness of the proposed method compared to the standard Frank-Wolfe algorithm.

Problem

• Optimizing structured nonconvex functions using Frank-Wolfe method
• Improving computational efficiency via DC decomposition selection
• Achieving first-order stationary points with reduced gradient calls

Innovation

• Projection-free Frank-Wolfe for nonconvex optimization
• DC decomposition enhances computational efficiency
• Gradient-efficient variant reduces gradient oracle calls