Society of Mind Meets Real-Time Strategy: A Hierarchical Multi-Agent Framework for Strategic Reasoning

2025-08-08

AI Summary

Large language models (LLMs) exhibit limited decision-making capability in long-horizon, partially observable, resource-constrained dynamic environments—such as real-time strategy (RTS) games—due to insufficient strategic reasoning and operational coordination. Method: This paper proposes HIMA, a hierarchical multi-agent framework featuring a meta-controller—termed the Strategic Planner—that dynamically aggregates and reconciles policy recommendations from specialized imitation-learning-based agents (e.g., air support, defense). HIMA integrates structured action sequence generation with a hierarchical control architecture to align low-level actions with high-level strategic objectives. Results: Evaluated on TEXTSCII-ALL, a full-race RTS benchmark, HIMA achieves significant improvements in strategic coherence, environmental adaptability, and computational efficiency. It outperforms existing LLM-based approaches across multiple dimensions, demonstrating superior end-to-end performance in complex, dynamic decision-making scenarios.

Technology Category

• Multiagent Systems: Multiagent Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large Language Models (LLMs) have recently demonstrated impressive action sequence prediction capabilities but often struggle with dynamic, long-horizon tasks such as real-time strategic games. In a game such as StarCraftII (SC2), agents need to manage resource constraints and adapt to evolving battlefield situations in a partially observable environment. This often overwhelms exisiting LLM-based approaches. To address these challenges, we propose a hierarchical multi-agent framework that employs specialized imitation learning agents under a meta-controller called Strategic Planner (SP). By expert demonstrations, each specialized agent learns a distinctive strategy, such as aerial support or defensive maneuvers, and produces coherent, structured multistep action sequences. The SP then orchestrates these proposals into a single, environmentally adaptive plan that ensures local decisions aligning with long-term strategies. We call this HIMA (Hierarchical Imitation Multi-Agent). We also present TEXTSCII-ALL, a comprehensive SC2 testbed that encompasses all race match combinations in SC2. Our empirical results show that HIMA outperforms state of the arts in strategic clarity, adaptability, and computational efficiency, underscoring the potential of combining specialized imitation modules with meta-level orchestration to develop more robust, general-purpose AI agents.

Problem

• LLMs struggle with dynamic long-horizon strategic tasks
• Agents face resource constraints in partially observable environments
• Existing LLM-based approaches lack adaptability in real-time strategy games

Innovation

• Hierarchical multi-agent framework for strategic reasoning
• Specialized imitation learning agents with expert demonstrations
• Meta-controller orchestrates adaptive long-term strategies