Decision-Driven Regularization: A Blended Model for Learning and Optimization

2026-08-15

AI Summary

This study addresses decision failures caused by predictive overfitting in contextual optimization by proposing a decision-driven regularized bi-objective framework. The method employs surrogate functions to resolve cost ambiguity, effectively balancing prediction accuracy with decision costs while unifying robust optimization and regret minimization perspectives and generalizing models such as SPO+. Experimental results on synthetic datasets demonstrate that the proposed framework significantly outperforms baselines including OLS, XGBoost, and SPO+, thereby enhancing both robustness and economic efficiency in end-to-end decision-making. Ultimately, this work establishes a novel paradigm for the synergistic integration of prediction and optimization.

Abstract

In contextual optimization, the decision-maker seeks optimal decisions to minimize a cost function, that varies based on observed features. This context is common in many business applications ranging from on-demand delivery and retail operations to portfolio optimization and inventory management. In this paper, we study the learning and optimization approach, which first learns how outcomes result from the features, and then selects optimal decisions based on these outcomes. We focus on the integrated learning and optimization literature, and identify that a lack of control for prediction accuracy can lead to overfitting and a loss of decision effectiveness against simple separate learning and optimization models. Instead, we propose a bi-objective formulation that balances prediction accuracy and cost minimization, termed decision-driven regularization. It also addresses ambiguity in the definition of the cost function via a surrogate that depends on a new hyperparameter. We additionally show that alternative perspectives for formulating the problem, namely robust optimization and regret minimization, lead to models that are closely related to our proposed model. As a consequence, our framework generalizes models such as SPO+. Our model is shown to be numerically superior to other benchmarks, such as OLS, Random Forest, XGBoost, SPO+, Perturbation Gradient, and Learning and Rank, in our synthetic studies.

Problem

• Contextual Optimization
• Integrated Learning and Optimization
• Overfitting
• Decision Effectiveness
• Cost Function Ambiguity

Innovation

• Decision-Driven Regularization
• Integrated Learning and Optimization
• Bi-objective Formulation
• Surrogate Cost Function
• Contextual Optimization