Scalable Decision-Focused Learning through Cost-Sensitive Regression

2026-05-18

AI Summary

This work addresses the high computational cost and poor scalability of traditional decision-focused learning, which requires repeatedly solving combinatorial optimization problems during training. The authors reformulate the problem as a cost-sensitive multi-output regression task and introduce a novel loss function that incorporates cost-agnostic normalization, decision-aware asymmetric prediction penalties, and instance-based cost modeling. This approach eliminates the need for repeated calls to an optimization solver, enabling end-to-end training. The proposed method achieves training efficiency gains substantial enough to enable, for the first time, scalable decision-focused learning on large-scale combinatorial optimization problems, while maintaining downstream task performance on par with existing approaches.

Abstract

Many real-world combinatorial problems involve uncertain parameters, which can be predicted given contextual features and historical data. These `predict-then-optimize' or `contextual optimization' problems have gained significant attention: end-to-end training methods can now minimize the downstream task cost rather than the predictive error. However, despite their effectiveness, these decision-focused learning (DFL) approaches often rely on repeated solving of the underlying combinatorial optimization problem during training, making them computationally expensive and difficult to scale. We reframe the learning problem as a cost-sensitive multi-output regression problem: multi-output due to the combinatorial problem having multiple uncertain parameters, and cost-sensitive due to the downstream task cost being the real target. Our technical contribution is the formalization of multiple loss function components that follow from this reframing: cost-insensitive normalization, decision-aware asymmetric penalization of over- and underpredictions, and instance-based costs that mimic the true downstream task-based loss locally. These components require zero or one solve per training data instance, while requiring no further solves during training. Experiments show that the combination of loss components achieves comparable downstream task quality to the state of the art, while being significantly more efficient, enabling scaling to problem sizes that have not been tackled before with DFL.

Problem

• Decision-Focused Learning
• Combinatorial Optimization
• Scalability
• Cost-Sensitive Regression
• Contextual Optimization

Innovation

• decision-focused learning
• cost-sensitive regression
• combinatorial optimization
• predict-then-optimize
• scalable learning