Optimal Model Selection for Conformalized Robust Optimization

2025-07-07

AI Summary

To address the strong model selection dependency and the trade-off between robustness and decision risk in Contextual Robust Optimization (CRO), this paper proposes the first model selection framework integrating conformal prediction with CRO. Methodologically, we introduce two novel frameworks—CROMS (automated) and CROiMS (personalized)—which pioneer covariate-aware conditional decision risk minimization. We further develop E-CROMS for computational efficiency and F-CROMS with finite-sample theoretical guarantees, jointly achieving asymptotic conditional robustness and decision efficiency. Empirically, extensive experiments on multiple synthetic and real-world datasets demonstrate that our approach significantly outperforms existing baselines, validating the critical role of principled model selection in enhancing both the efficacy and reliability of downstream decision-making.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Intelligent Robots: Learning & Optimization for ROB
• Constraint Satisfaction and Optimization: Constraint Optimization

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

In decision-making under uncertainty, Contextual Robust Optimization (CRO) provides reliability by minimizing the worst-case decision loss over a prediction set, hedging against label variability. While recent advances use conformal prediction to construct prediction sets for machine learning models, the downstream decisions critically depend on model selection. This paper introduces novel model selection frameworks for CRO that unify robustness control with decision risk minimization. We first propose Conformalized Robust Optimization with Model Selection (CROMS), which automatically selects models to approximately minimize the average decision risk in CRO solutions. We develop two algorithms: E-CROMS, which is computationally efficient, and F-CROMS, which enjoys a marginal robustness guarantee in finite samples. Further, we introduce Conformalized Robust Optimization with Individualized Model Selection (CROiMS), which performs individualized model selection by minimizing the conditional decision risk given the covariate of test data. This framework advances conformal prediction methodology by enabling covariate-aware model selection. Theoretically, CROiMS achieves asymptotic conditional robustness and decision efficiency under mild assumptions. Numerical results demonstrate significant improvements in decision efficiency and robustness across diverse synthetic and real-world applications, outperforming baseline approaches.

Problem

• Selecting optimal models for robust decision-making under uncertainty
• Unifying robustness control with decision risk minimization
• Enabling covariate-aware model selection for improved efficiency

Innovation

• Unifies robustness control with decision risk minimization
• Introduces CROMS for automatic model selection
• Achieves conditional robustness with CROiMS