A Quadratic Link between Out-of-Sample $R^2$ and Directional Accuracy

2026-02-08

AI Summary

This study addresses the frequent disconnect between out-of-sample R² and directional accuracy in financial time series forecasting, where R² often approaches zero or becomes negative despite low directional accuracy. The authors derive, for the first time, an analytical quadratic relationship between these two metrics under mean squared error (MSE)-optimal predictions, revealing their intrinsic connection. By employing a random walk benchmark and incorporating assumptions of sign correctness and magnitude independence, they demonstrate that R² naturally converges to zero when directional accuracy is insufficient. Moreover, they show that negative R² values are theoretically justified under suboptimal models or finite-sample settings. This work provides a rigorous theoretical foundation for evaluating predictive performance and bridges the interpretive gap between these commonly used but seemingly discordant evaluation metrics.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

This study provides a novel perspective on the metric disconnect phenomenon in financial time series forecasting through an analytical link that reconciles the out-of-sample $R^2$ ($R^2_{OOS}$) and directional accuracy (DA). In particular, using the random walk model as a baseline and assuming that sign correctness is independent of realized magnitude, we show that these two metrics exhibit a quadratic relationship for MSE-optimal point forecasts. For point forecasts with modest DA, the theoretical value of $R^2_{OOS}$ is intrinsically negligible. Thus, a negative empirical $R^2_{OOS}$ is expected if the model is suboptimal or affected by finite sample noise.

Problem

• financial time series forecasting
• metric disconnect
• out-of-sample R²
• directional accuracy
• forecast evaluation

Innovation

• out-of-sample R²
• directional accuracy
• quadratic relationship
• financial time series forecasting
• metric disconnect