MARS: A Meta-Adaptive Reinforcement Learning Framework for Risk-Aware Multi-Agent Portfolio Management

2025-08-01

AI Summary

To address the challenge of jointly optimizing risk and return in automated portfolio management under dynamic market conditions, this paper proposes a multi-agent reinforcement learning framework. Methodologically, it employs a two-tier heterogeneous architecture: a lower tier comprising agents with diverse risk preferences to capture market cyclicity through behavioral diversity; and an upper tier integrating a Meta-Adaptive Controller (MAC) and a Safety-Critic network to enable risk-aware dynamic asset allocation and policy scheduling. Empirical evaluation across major international equity indices—including crisis periods—demonstrates that the approach significantly reduces maximum drawdown (−32.1% on average) and volatility (−24.7% on average), while sustaining robust returns. The core contribution lies in the principled integration of risk-preference heterogeneity, meta-level adaptive regulation, and safety-constrained optimization, thereby enhancing the robustness and adaptability of investment strategies in uncertain environments.

Technology Category

• Multiagent Systems: Adversarial Agents
• Intelligent Robots: Learning & Optimization for ROB
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Reinforcement Learning (RL) has shown significant promise in automated portfolio management; however, effectively balancing risk and return remains a central challenge, as many models fail to adapt to dynamically changing market conditions. In this paper, we propose Meta-controlled Agents for a Risk-aware System (MARS), a novel RL framework designed to explicitly address this limitation through a multi-agent, risk-aware approach. Instead of a single monolithic model, MARS employs a Heterogeneous Agent Ensemble where each agent possesses a unique, intrinsic risk profile. This profile is enforced by a dedicated Safety-Critic network and a specific risk-tolerance threshold, allowing agents to specialize in behaviors ranging from capital preservation to aggressive growth. To navigate different market regimes, a high-level Meta-Adaptive Controller (MAC) learns to dynamically orchestrate the ensemble. By adjusting its reliance on conservative versus aggressive agents, the MAC effectively lowers portfolio volatility during downturns and seeks higher returns in bull markets, thus minimizing maximum drawdown and enhancing overall stability. This two-tiered structure allows MARS to generate a disciplined and adaptive portfolio that is robust to market fluctuations. The framework achieves a superior balance between risk and return by leveraging behavioral diversity rather than explicit market-feature engineering. Experiments on major international stock indexes, including periods of significant financial crisis, demonstrate the efficacy of our framework on risk-adjusted criteria, significantly reducing maximum drawdown and volatility while maintaining competitive returns.

Problem

• Balancing risk and return in dynamic markets
• Adapting to changing market conditions effectively
• Minimizing portfolio volatility and drawdowns

Innovation

• Heterogeneous Agent Ensemble with unique risk profiles
• Meta-Adaptive Controller for dynamic agent orchestration
• Safety-Critic network enforcing risk-tolerance thresholds