Smart Predict--then--Optimize Paradigm for Portfolio Optimization in Real Markets

2026-01-07

AI Summary

This study addresses the limitation that improving traditional return prediction accuracy often fails to enhance portfolio decisions, particularly in markets with transaction costs and realistic constraints. The authors propose a decision-focused approach based on the Smart Predict-then-Optimize (SPO) paradigm, which directly aligns the training objective of the prediction model with portfolio optimization performance rather than minimizing point prediction error alone. The method integrates a linear predictor, technical indicators, and an SPO-based surrogate loss, while explicitly modeling transaction costs, turnover control, and regularization within the optimization layer. Rolling backtests on U.S. ETF data from 2015 to 2025 demonstrate that the approach significantly improves risk-adjusted returns and exhibits robustness during extreme market conditions, such as the 2020 pandemic. This work provides the first empirical validation of the SPO framework’s dual advantages in interpretability and out-of-sample performance in real-world trading environments.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Improvements in return forecast accuracy do not always lead to proportional improvements in portfolio decision quality, especially under realistic trading frictions and constraints. This paper adopts the Smart Predict--then--Optimize (SPO) paradigm for portfolio optimization in real markets, which explicitly aligns the learning objective with downstream portfolio decision quality rather than pointwise prediction accuracy. Within this paradigm, predictive models are trained using an SPO-based surrogate loss that directly reflects the performance of the resulting investment decisions. To preserve interpretability and robustness, we employ linear predictors built on return-based and technical-indicator features and integrate them with portfolio optimization models that incorporate transaction costs, turnover control, and regularization. We evaluate the proposed approach on U.S. ETF data (2015--2025) using a rolling-window backtest with monthly rebalancing. Empirical results show that decision-focused training consistently improves risk-adjusted performance over predict--then--optimize baselines and classical optimization benchmarks, and yields strong robustness during adverse market regimes (e.g., the 2020 COVID-19). These findings highlight the practical value of the Smart Predict--then--Optimize paradigm for portfolio optimization in realistic and non-stationary financial environments.

Problem

• portfolio optimization
• decision-focused learning
• trading frictions
• predict-then-optimize
• financial markets

Innovation

• Smart Predict-then-Optimize
• decision-focused learning
• portfolio optimization
• transaction costs
• robustness