SAMP-HDRL: Segmented Allocation with Momentum-Adjusted Utility for Multi-agent Portfolio Management via Hierarchical Deep Reinforcement Learning

2025-12-28

AI Summary

Non-stationary financial markets pose challenges including abrupt regime shifts, time-varying asset correlations, and poor interpretability of deep reinforcement learning (DRL) policies. Method: We propose a hierarchical DRL framework: an upper-level agent identifies global market regimes, while a lower-level agent dynamically clusters assets via time-series clustering and performs constrained portfolio allocation; risk-free and risky asset allocation is jointly optimized through a momentum-adjusted utility function. Contribution/Results: Our approach innovatively integrates three mechanisms—dynamic asset grouping, hierarchical decision-making, and utility-driven capital allocation—while explicitly embedding structural market constraints to yield interpretable “diversified-yet-focused” strategies. Evaluated across three distinct market regimes (2019–2021), it consistently outperforms nine baseline methods, achieving average improvements of ≥5% in annualized return, Sharpe ratio, and Sortino ratio, and +2% in Omega ratio—with particularly pronounced gains during volatile periods.

Technology Category

• Machine Learning: Reinforcement Learning
• Reasoning under Uncertainty: Stochastic Optimization
• Multiagent Systems: Mechanism Design

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Portfolio optimization in non-stationary markets is challenging due to regime shifts, dynamic correlations, and the limited interpretability of deep reinforcement learning (DRL) policies. We propose a Segmented Allocation with Momentum-Adjusted Utility for Multi-agent Portfolio Management via Hierarchical Deep Reinforcement Learning (SAMP-HDRL). The framework first applies dynamic asset grouping to partition the market into high-quality and ordinary subsets. An upper-level agent extracts global market signals, while lower-level agents perform intra-group allocation under mask constraints. A utility-based capital allocation mechanism integrates risky and risk-free assets, ensuring coherent coordination between global and local decisions. backtests across three market regimes (2019--2021) demonstrate that SAMP-HDRL consistently outperforms nine traditional baselines and nine DRL benchmarks under volatile and oscillating conditions. Compared with the strongest baseline, our method achieves at least 5% higher Return, 5% higher Sharpe ratio, 5% higher Sortino ratio, and 2% higher Omega ratio, with substantially larger gains observed in turbulent markets. Ablation studies confirm that upper--lower coordination, dynamic clustering, and capital allocation are indispensable to robustness. SHAP-based interpretability further reveals a complementary ``diversified + concentrated'' mechanism across agents, providing transparent insights into decision-making. Overall, SAMP-HDRL embeds structural market constraints directly into the DRL pipeline, offering improved adaptability, robustness, and interpretability in complex financial environments.

Problem

• Addresses portfolio optimization in non-stationary markets with regime shifts and dynamic correlations.
• Enhances interpretability of deep reinforcement learning policies for multi-agent portfolio management.
• Integrates global market signals with local asset allocation under structural constraints for robustness.

Innovation

• Hierarchical DRL with upper-lower agent coordination
• Dynamic asset grouping into high-quality and ordinary subsets
• Utility-based capital allocation integrating risky and risk-free assets