Heterogeneous RBCs via deep multi-agent reinforcement learning

2025-10-14

AI Summary

Existing macroeconomic models face a trade-off in handling agent heterogeneity: general-equilibrium frameworks (e.g., HANK, Krusell–Smith) rely on unrealistic rational expectations and incur prohibitive computational costs, limiting their capacity for rich heterogeneity; conversely, rule-based agent-based models (ABMs) offer flexibility but lack theoretical foundations and require extensive ad hoc calibration. Method: This paper introduces MARL-BC—a novel framework integrating deep multi-agent reinforcement learning (MARL) into business-cycle modeling—where individual decision rules emerge endogenously via function approximation and policy gradient methods, reconciling rational-expectations consistency with large-scale heterogeneous interactions. Contribution/Results: MARL-BC unifies ABM and general-equilibrium paradigms. It replicates canonical RBC and Krusell–Smith results while enabling simulation of high-dimensional heterogeneous systems previously intractable to conventional methods, thereby substantially enhancing macroeconomic models’ explanatory power and scalability.

Technology Category

• Multiagent Systems: Multiagent Learning
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Game Theory and Economic Paradigms: Behavioral Game Theory

Application Category

• Economics, Online Markets and Human Computation: Sustainability of Web economics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Current macroeconomic models with agent heterogeneity can be broadly divided into two main groups. Heterogeneous-agent general equilibrium (GE) models, such as those based on Heterogeneous Agents New Keynesian (HANK) or Krusell-Smith (KS) approaches, rely on GE and 'rational expectations', somewhat unrealistic assumptions that make the models very computationally cumbersome, which in turn limits the amount of heterogeneity that can be modelled. In contrast, agent-based models (ABMs) can flexibly encompass a large number of arbitrarily heterogeneous agents, but typically require the specification of explicit behavioural rules, which can lead to a lengthy trial-and-error model-development process. To address these limitations, we introduce MARL-BC, a framework that integrates deep multi-agent reinforcement learning (MARL) with Real Business Cycle (RBC) models. We demonstrate that MARL-BC can: (1) recover textbook RBC results when using a single agent; (2) recover the results of the mean-field KS model using a large number of identical agents; and (3) effectively simulate rich heterogeneity among agents, a hard task for traditional GE approaches. Our framework can be thought of as an ABM if used with a variety of heterogeneous interacting agents, and can reproduce GE results in limit cases. As such, it is a step towards a synthesis of these often opposed modelling paradigms.

Problem

• Addressing computational limitations in heterogeneous-agent macroeconomic models
• Eliminating reliance on unrealistic rational expectations assumptions
• Enabling flexible modeling of rich agent heterogeneity

Innovation

• Integrates deep multi-agent reinforcement learning with RBC models
• Simulates rich heterogeneity among agents effectively
• Synthesizes agent-based and general equilibrium modeling approaches