MARLIN: Multi-Agent Reinforcement Learning for Incremental DAG Discovery

2026-03-18

AI Summary

This work addresses the challenge of efficiently discovering causal structures—represented as directed acyclic graphs (DAGs)—in online settings by proposing an incremental DAG discovery method based on multi-agent reinforcement learning. The approach employs a generative strategy that maps continuous spaces to DAGs and introduces a dual-agent collaborative mechanism comprising both state-dependent and state-independent agents. To enable efficient parallelized incremental causal inference, the method further incorporates a factorized action space. Experimental evaluations on both synthetic and real-world datasets demonstrate that the proposed method significantly outperforms existing approaches in terms of both accuracy and computational efficiency, thereby substantially enhancing the online applicability of causal discovery.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Multiagent Systems: Adversarial Agents

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Uncovering causal structures from observational data is crucial for understanding complex systems and making informed decisions. While reinforcement learning (RL) has shown promise in identifying these structures in the form of a directed acyclic graph (DAG), existing methods often lack efficiency, making them unsuitable for online applications. In this paper, we propose MARLIN, an efficient multi agent RL based approach for incremental DAG learning. MARLIN uses a DAG generation policy that maps a continuous real valued space to the DAG space as an intra batch strategy, then incorporates two RL agents state specific and state invariant to uncover causal relationships and integrates these agents into an incremental learning framework. Furthermore, the framework leverages a factored action space to enhance parallelization efficiency. Extensive experiments on synthetic and real datasets demonstrate that MARLIN outperforms state of the art methods in terms of both efficiency and effectiveness.

Problem

• causal discovery
• directed acyclic graph
• reinforcement learning
• online learning
• efficiency

Innovation

• Multi-Agent Reinforcement Learning
• Incremental DAG Discovery
• Causal Structure Learning
• Factored Action Space
• Online Causal Inference