Contextual Strongly Convex Simulation Optimization: Optimize then Predict with Inexact Solutions

2025-12-05

AI Summary

This paper investigates how bias and variance induced by offline approximation of the optimal solution function affect the optimality gap in contextual strongly convex simulation optimization. We propose an “optimize-then-predict” framework, employing Polyak–Ruppert averaged stochastic gradient descent as the base optimizer and integrating four smoothing techniques—k-nearest neighbors, kernel smoothing, linear regression, and kernel ridge regression—to construct the solution function. We establish, for the first time, a unified statistical analysis framework that explicitly quantifies the relationship between the estimator’s statistical error and the online decision’s optimality gap. Furthermore, we derive the optimal allocation of computational budget between covariate coverage number and per-point simulation replications. Theoretically, the optimality gap achieves a near-optimal Γ⁻¹ convergence rate; numerical experiments corroborate both the theoretical findings and the practical efficacy of the proposed approach.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Optimization

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In this work, we study contextual strongly convex simulation optimization and adopt an "optimize then predict" (OTP) approach for real-time decision making. In the offline stage, simulation optimization is conducted across a set of covariates to approximate the optimal-solution function; in the online stage, decisions are obtained by evaluating this approximation at the observed covariate. The central theoretical challenge is to understand how the inexactness of solutions generated by simulation-optimization algorithms affects the optimality gap, which is overlooked in existing studies. To address this, we develop a unified analysis framework that explicitly accounts for both solution bias and variance. Using Polyak-Ruppert averaging SGD as an illustrative simulation-optimization algorithm, we analyze the optimality gap of OTP under four representative smoothing techniques: $k$ nearest neighbor, kernel smoothing, linear regression, and kernel ridge regression. We establish convergence rates, derive the optimal allocation of the computational budget $Γ$ between the number of design covariates and the per-covariate simulation effort, and demonstrate the convergence rate can approximately achieve $Γ^{-1}$ under appropriate smoothing technique and sample-allocation rule. Finally, through a numerical study, we validate the theoretical findings and demonstrate the effectiveness and practical value of the proposed approach.

Problem

• Optimizes contextual simulation with inexact solutions
• Analyzes optimality gap from solution bias and variance
• Determines computational budget allocation for convergence rates

Innovation

• Optimize then predict approach for real-time decisions
• Unified analysis framework for solution bias and variance
• Optimal computational budget allocation between covariates and simulation