A Practical Machine Learning Approach for Dynamic Stock Recommendation

2025-11-15

AI Summary

Persistent challenges in dynamic stock selection for S&P 500 constituents—namely, the inability of single-strategy approaches to sustain robust outperformance and the scalability limitations of manual coverage—motivate this work. Method: We propose an automated stock-selection framework based on rolling time windows and dynamic model selection. It integrates multiple regression models—including linear regression, ridge regression, stepwise regression, random forest, and gradient boosting—and selects the optimal model each period based on out-of-sample prediction error. Selected models generate stock-level expected return rankings, which feed into mean-variance and minimum-variance portfolio optimization to construct long–short portfolios. Contribution/Results: By embedding adaptive model selection directly into the rolling forecasting pipeline, our framework significantly enhances responsiveness to market regime shifts. Empirical results demonstrate consistent outperformance over the S&P 500 long-only benchmark in both Sharpe ratio and cumulative returns, confirming its robustness and practical deployability.

Technology Category

• Machine Learning: Ensemble Methods
• Search and Optimization: Mixed Discrete/Continuous Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Stock recommendation is vital to investment companies and investors. However, no single stock selection strategy will always win while analysts may not have enough time to check all S&P 500 stocks (the Standard & Poor's 500). In this paper, we propose a practical scheme that recommends stocks from S&P 500 using machine learning. Our basic idea is to buy and hold the top 20% stocks dynamically. First, we select representative stock indicators with good explanatory power. Secondly, we take five frequently used machine learning methods, including linear regression, ridge regression, stepwise regression, random forest and generalized boosted regression, to model stock indicators and quarterly log-return in a rolling window. Thirdly, we choose the model with the lowest Mean Square Error in each period to rank stocks. Finally, we test the selected stocks by conducting portfolio allocation methods such as equally weighted, mean-variance, and minimum-variance. Our empirical results show that the proposed scheme outperforms the long-only strategy on the S&P 500 index in terms of Sharpe ratio and cumulative returns. This work is fully open-sourced at href{https://github.com/AI4Finance-Foundation/Dynamic-Stock-Recommendation-Machine_Learning-Published-Paper-IEEE}{GitHub}.

Problem

• Recommending top S&P 500 stocks dynamically using machine learning models
• Overcoming limitations of static strategies and human analyst capacity constraints
• Selecting optimal models through rolling window evaluation of stock indicators

Innovation

• Dynamically selects top 20% stocks using rolling window
• Applies five machine learning models to rank stocks
• Chooses best model by minimizing Mean Square Error