Causal Effects with Unobserved Unit Types in Interacting Human-AI Systems

2026-03-01

AI Summary

This study addresses the challenge of identifying human-specific causal effects in mixed human–AI interaction systems, where individual identities and interaction networks are typically unobserved. The authors propose a novel approach that leverages only the known group-level distribution of human composition, combining subgroup-average human proportions with variation in treatment exposure to consistently identify causal effects attributable exclusively to humans. By embedding this strategy within a Causal Message Passing (CMP) framework to model outcome dynamics, the method enables consistent estimation of human-specific effects for the first time in such settings. Empirical validation is conducted using a simulation platform populated by behaviorally heterogeneous AI agents powered by large language models, demonstrating that the proposed approach accurately recovers human-specific causal effects even in complex, mixed-agent environments.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality

Application Category

• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We study experiments on interacting populations of humans and AI agents, where both unit types and the interaction network remain unobserved. Although causal effects propagate throughout the system, the goal is to estimate effects on humans. Examples include online platforms where human users interact alongside AI-driven accounts. We assume a human-AI prior that gives each unit a probability of being human. While humans cannot be distinguished at the unit level, the prior allows us to compute the average human composition within large subpopulations. We then model outcome dynamics through a causal message passing (CMP) framework and analyze sample-mean outcomes across subpopulations. We show that by constructing subpopulations that vary in expected human composition and treatment exposure, one can consistently recover human-specific causal effects. Our results characterize when distributional knowledge of population composition (without observing unit types or the interaction network) is sufficient for identification. We validate the approach on a simulated human-AI platform driven by behaviorally differentiated LLM agents. Together, these results provide a theoretical and practical framework for experimentation in emerging human-AI systems.

Problem

• causal effects
• unobserved unit types
• human-AI interaction
• treatment effect estimation
• interacting populations

Innovation

• causal message passing
• human-AI systems
• unobserved unit types
• population composition prior
• heterogeneous treatment effects