GPU-friendly and Linearly Convergent First-order Methods for Certifying Optimal $k$-sparse GLMs

2026-03-01

AI Summary

This work addresses the computational challenges in verifying optimality for sparse generalized linear models under cardinality constraints, where existing methods are prohibitively expensive and poorly scalable. By reformulating the perspective relaxation as a composite optimization problem, the authors establish a connection between primal quadratic growth and dual quadratic decay. They propose a duality-gap-based restarting strategy that upgrades several sublinearly convergent algorithms to linear convergence and design a dedicated routine operating in log-linear time to compute the perspective regularizer and its proximal operator exactly. Coupled with GPU-accelerated matrix-vector operations, the proposed framework achieves several orders of magnitude speedup in dual bound computation on both synthetic and real-world datasets, substantially enhancing the scalability of branch-and-bound methods for large-scale instances.

Technology Category

• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

We investigate the problem of certifying optimality for sparse generalized linear models (GLMs), where sparsity is enforced through a cardinality constraint. While Branch-and-Bound (BnB) frameworks can certify optimality using perspective relaxations, existing methods for solving these relaxations are computationally intensive, limiting their scalability. To address this challenge, we reformulate the relaxations as composite optimization problems and develop a unified proximal framework that is both linearly convergent and computationally efficient. Under specific geometric regularity conditions, our analysis links primal quadratic growth to dual quadratic decay, yielding error bounds that make the Fenchel duality gap a sharp proxy for progress towards the solution set. This leads to a duality gap-based restart scheme that upgrades a broad class of sublinear proximal methods to provably linearly convergent methods, and applies beyond the sparse GLM setting. For the implicit perspective regularizer, we further derive specialized routines to evaluate the regularizer and its proximal operator exactly in log-linear time, avoiding costly generic conic solvers. The resulting iterations are dominated by matrix--vector multiplications, which enables GPU acceleration. Experiments on synthetic and real-world datasets show orders-of-magnitude faster dual-bound computations and substantially improved BnB scalability on large instances.

Problem

• sparse GLMs
• optimality certification
• cardinality constraint
• perspective relaxation
• computational scalability

Innovation

• linear convergence
• proximal methods
• perspective relaxation
• GPU acceleration
• Fenchel duality gap