Two Completely Parameter-Free Alternating Gradient Projection Algorithms for Nonconvex-(strongly) Concave Minimax Problems

2024-07-31

arXiv.org

AI Summary

This work addresses nonconvex-(strongly) concave minimax optimization problems by proposing two fully parameter-free alternating gradient projection algorithms—PF-AGP-NSC and PF-AGP-NC—that require no prior knowledge of Lipschitz constants, strong concavity moduli, or other problem-specific parameters. Both methods employ adaptive backtracking line search to eliminate hyperparameter tuning. Theoretically, PF-AGP-NSC achieves an $varepsilon$-stationary point in $O(Lkappa^3varepsilon^{-2})$ gradient evaluations under the nonconvex-strongly concave setting, while PF-AGP-NC attains the same guarantee in $O(L^4varepsilon^{-4})$ gradient calls under the nonconvex-concave setting—matching the optimal complexity rates for their respective regimes. Numerical experiments demonstrate robustness and efficiency across diverse benchmarks. To the best of our knowledge, these are the first fully parameter-free, single-loop algorithms applicable to both nonconvex-strongly concave and nonconvex-concave minimax problems.

Technology Category

• Search and Optimization: Non-convex Optimization
• Multiagent Systems: Mechanism Design
• Machine Learning: Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

Due to their importance in various emerging applications, efficient algorithms for solving minimax problems have recently received increasing attention. However, many existing algorithms require prior knowledge of the problem parameters in order to achieve optimal iteration complexity. In this paper, we propose two completely parameter-free alternating gradient projection algorithms, i.e., the PF-AGP-NSC algorithm and the PF-AGP-NC algorithm, to solve the smooth nonconvex-strongly concave and nonconvex-concave minimax problems respectively using a backtracking strategy, which does not require prior knowledge of parameters such as the Lipschtiz constant $L$ or the strongly concave constant $mu$. Moreover, we show that the total number of gradient calls of the PF-AGP-NSC algorithm and the PF-AGP-NC algorithm to obtain an $varepsilon$-stationary point is upper bounded by $mathcal{O}left( Lkappa^3varepsilon^{-2} ight)$ and $mathcal{O}left( L^4varepsilon^{-4} ight)$ respectively, where $kappa$ is the condition number. As far as we know, the PF-AGP-NSC algorithm and the PF-AGP-NC algorithm are the first completely parameter-free algorithms for solving nonconvex-strongly concave minimax problems and nonconvex-concave minimax problems respectively. Numerical results validate the efficiency of the proposed PF-AGP algorithm.

Problem

• Develop parameter-free algorithms for nonconvex minimax problems
• Eliminate need for prior knowledge of problem parameters
• Achieve optimal iteration complexity without Lipschitz or modulus info

Innovation

• Parameter-free single-loop algorithms for minimax problems
• Line search without prior knowledge of parameters
• Optimal gradient call complexity for stationary points