HARLF: Hierarchical Reinforcement Learning and Lightweight LLM-Driven Sentiment Integration for Financial Portfolio Optimization

2025-07-24

AI Summary

This study addresses the challenges of integrating financial news sentiment signals with traditional market indicators and enhancing investment decision stability. We propose a lightweight large language model (LLM)–enhanced, hierarchical deep reinforcement learning (DRL) framework comprising three cooperative agents—base, meta, and super agents—and design a cross-modal, scalable fusion mechanism to enable dynamic alignment of heterogeneous, multi-source information and hierarchical decision-making. The approach ensures interpretability, robustness, and open-source reproducibility. Empirical evaluation over 2018–2024 demonstrates that the strategy achieves a 26% annualized return and a Sharpe ratio of 1.2, significantly outperforming both an equally weighted portfolio and the S&P 500 benchmark. These results validate the efficacy of joint sentiment–market modeling in improving risk-adjusted returns and decision stability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Multiagent Systems: Multiagent Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

This paper presents a novel hierarchical framework for portfolio optimization, integrating lightweight Large Language Models (LLMs) with Deep Reinforcement Learning (DRL) to combine sentiment signals from financial news with traditional market indicators. Our three-tier architecture employs base RL agents to process hybrid data, meta-agents to aggregate their decisions, and a super-agent to merge decisions based on market data and sentiment analysis. Evaluated on data from 2018 to 2024, after training on 2000-2017, the framework achieves a 26% annualized return and a Sharpe ratio of 1.2, outperforming equal-weighted and S&P 500 benchmarks. Key contributions include scalable cross-modal integration, a hierarchical RL structure for enhanced stability, and open-source reproducibility.

Problem

• Optimize financial portfolios using hierarchical reinforcement learning
• Integrate sentiment analysis from news with market data
• Improve investment returns and stability via multi-agent architecture

Innovation

• Hierarchical RL for portfolio optimization
• Lightweight LLM-driven sentiment integration
• Three-tier agent architecture