AlphaQuanter: An End-to-End Tool-Orchestrated Agentic Reinforcement Learning Framework for Stock Trading

2025-10-15

AI Summary

Existing LLM-based agents for automated stock trading suffer from low efficiency, inconsistent trading signals, absence of end-to-end optimization, and weak capability to learn from market feedback. This paper proposes the first end-to-end single-agent reinforcement learning framework for autonomous trading, integrating large language models (LLMs), tool-augmented decision-making, proactive information retrieval, and dynamic policy updating—yielding an interpretable, traceable, and auditable trading workflow. Innovatively, we introduce a transparent tool-coordination architecture and a single-agent dynamic policy mechanism to eliminate multi-agent signal conflicts, ensuring policy consistency and full traceability of reasoning steps. Empirical evaluation demonstrates state-of-the-art performance on key financial metrics—including annualized return and Sharpe ratio—while interpretability analysis uncovers sophisticated, human-informative market adaptation strategies.

Technology Category

• Machine Learning: Reinforcement Learning
• Multiagent Systems: Mechanism Design
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

While Large Language Model (LLM) agents show promise in automated trading, they still face critical limitations. Prominent multi-agent frameworks often suffer from inefficiency, produce inconsistent signals, and lack the end-to-end optimization required to learn a coherent strategy from market feedback. To address this, we introduce AlphaQuanter, a single-agent framework that uses reinforcement learning (RL) to learn a dynamic policy over a transparent, tool-augmented decision workflow, which empowers a single agent to autonomously orchestrate tools and proactively acquire information on demand, establishing a transparent and auditable reasoning process. Extensive experiments demonstrate that AlphaQuanter achieves state-of-the-art performance on key financial metrics. Moreover, its interpretable reasoning reveals sophisticated strategies, offering novel and valuable insights for human traders. Our code for data acquisition and agent training is publicly available at: https://github.com/AlphaQuanter/AlphaQuanter

Problem

• Addresses inefficiency in multi-agent trading frameworks
• Solves inconsistent trading signal generation issues
• Enables end-to-end strategy optimization from market feedback

Innovation

• Single-agent reinforcement learning for dynamic trading policy
• Tool-orchestrated workflow enabling autonomous information acquisition
• Transparent decision process providing interpretable trading strategies