IV Co-Scientist: Multi-Agent LLM Framework for Causal Instrumental Variable Discovery

2026-02-08

AI Summary

This work addresses the challenging problem of identifying valid instrumental variables (IVs) in causal inference under unmeasured confounding. It proposes the first collaborative framework based on multi-agent large language models, wherein agents iteratively propose, critique, and refine candidate instruments to automatically discover valid IVs for a given treatment–outcome pair. The approach innovatively introduces a label-free statistical consistency test to assess instrument validity without requiring ground-truth labels, complemented by a two-stage evaluation strategy for rigorous validation. Experimental results demonstrate that the system not only recovers known valid instruments and avoids invalid candidates but also exhibits promising potential in uncovering novel valid instruments from large-scale observational data.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Multiagent Systems: Mechanism Design

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In the presence of confounding between an endogenous variable and the outcome, instrumental variables (IVs) are used to isolate the causal effect of the endogenous variable. Identifying valid instruments requires interdisciplinary knowledge, creativity, and contextual understanding, making it a non-trivial task. In this paper, we investigate whether large language models (LLMs) can aid in this task. We perform a two-stage evaluation framework. First, we test whether LLMs can recover well-established instruments from the literature, assessing their ability to replicate standard reasoning. Second, we evaluate whether LLMs can identify and avoid instruments that have been empirically or theoretically discredited. Building on these results, we introduce IV Co-Scientist, a multi-agent system that proposes, critiques, and refines IVs for a given treatment-outcome pair. We also introduce a statistical test to contextualize consistency in the absence of ground truth. Our results show the potential of LLMs to discover valid instrumental variables from a large observational database.

Problem

• Instrumental Variables
• Causal Inference
• Confounding
• Endogenous Variable
• Causal Effect

Innovation

• Instrumental Variable Discovery
• Multi-Agent LLM
• Causal Inference
• Large Language Models
• Statistical Consistency Test