Revisiting Ensemble Methods for Stock Trading and Crypto Trading Tasks at ACM ICAIF FinRL Contest 2023-2024

2025-01-18

AI Summary

To address the dual challenges of policy instability and low sample efficiency in financial reinforcement learning, this paper proposes a robust ensemble trading framework enabling thousand-scale parallel simulation on a single GPU. Methodologically, it integrates multi-agent PPO/SAC training, policy averaging and voting-based ensemble mechanisms, domain-specific financial time-series feature engineering, and a risk-aware reward function. Its key innovation lies in the first-ever realization of over one thousand concurrent environment simulations on a single GPU, coupled with a novel risk-mitigating policy fusion paradigm tailored for high-volatility markets. Experiments demonstrate a 1746× speedup in sampling throughput; compared to single-agent baselines, the framework achieves higher cumulative returns, reduces maximum drawdown by 4.17%, improves the Sharpe ratio by 0.21, and significantly enhances generalization capability and decision stability.

Technology Category

• Machine Learning: Ensemble Methods
• Multiagent Systems: Adversarial Agents
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Reinforcement learning has demonstrated great potential for performing financial tasks. However, it faces two major challenges: policy instability and sampling bottlenecks. In this paper, we revisit ensemble methods with massively parallel simulations on graphics processing units (GPUs), significantly enhancing the computational efficiency and robustness of trained models in volatile financial markets. Our approach leverages the parallel processing capability of GPUs to significantly improve the sampling speed for training ensemble models. The ensemble models combine the strengths of component agents to improve the robustness of financial decision-making strategies. We conduct experiments in both stock and cryptocurrency trading tasks to evaluate the effectiveness of our approach. Massively parallel simulation on a single GPU improves the sampling speed by up to $1,746 imes$ using $2,048$ parallel environments compared to a single environment. The ensemble models have high cumulative returns and outperform some individual agents, reducing maximum drawdown by up to $4.17%$ and improving the Sharpe ratio by up to $0.21$. This paper describes trading tasks at ACM ICAIF FinRL Contests in 2023 and 2024.

Problem

• Reinforcement Learning
• Financial Markets
• Decision Instability

Innovation

• Parallel Processing
• Integrated Model Strategy
• Financial Decision Stability