Online Statistical Inference of Constrained Stochastic Optimization via Random Scaling

2025-05-23

AI Summary

We address constrained stochastic nonlinear optimization in large-scale online settings. To enable real-time statistical inference without storing historical data, we propose a novel random scaling-based online inference framework, integrated with Sketching-based Stochastic Sequential Quadratic Programming (SSQP). Our method operates solely on the primal-dual iterate sequence ((x_t, lambda_t)), eliminating matrix inversion and Hessian computation. The resulting test statistic admits a parameter-free limiting distribution—establishing, for the first time in constrained stochastic optimization, “matrix-free” asymptotically valid confidence intervals. We prove the asymptotic efficiency of the estimator under standard regularity conditions. Numerical experiments on nonlinear constrained regression demonstrate that our approach significantly improves both coverage accuracy and computational efficiency compared to existing online inference methods.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Non-convex Optimization
• Constraint Satisfaction and Optimization: Constraint Optimization

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Constrained stochastic nonlinear optimization problems have attracted significant attention for their ability to model complex real-world scenarios in physics, economics, and biology. As datasets continue to grow, online inference methods have become crucial for enabling real-time decision-making without the need to store historical data. In this work, we develop an online inference procedure for constrained stochastic optimization by leveraging a method called Sketched Stochastic Sequential Quadratic Programming (SSQP). As a direct generalization of sketched Newton methods, SSQP approximates the objective with a quadratic model and the constraints with a linear model at each step, then applies a sketching solver to inexactly solve the resulting subproblem. Building on this design, we propose a new online inference procedure called random scaling. In particular, we construct a test statistic based on SSQP iterates whose limiting distribution is free of any unknown parameters. Compared to existing online inference procedures, our approach offers two key advantages: (i) it enables the construction of asymptotically valid confidence intervals; and (ii) it is matrix-free, i.e. the computation involves only primal-dual SSQP iterates $(oldsymbol{x}_t, oldsymbol{lambda}_t)$ without requiring any matrix inversions. We validate our theory through numerical experiments on nonlinearly constrained regression problems and demonstrate the superior performance of our random scaling method over existing inference procedures.

Problem

• Online inference for constrained stochastic optimization problems
• Real-time decision-making without storing historical data
• Matrix-free confidence intervals via random scaling method

Innovation

• Uses Sketched Stochastic Sequential Quadratic Programming
• Applies random scaling for online inference
• Matrix-free method without matrix inversions