Causal Perception

2024-01-24

arXiv.org

AI Summary

This paper addresses perceptual bias arising from inconsistent human interpretations of model outputs in human-AI collaborative decision-making. We propose the first causal perception framework, formalizing “perception” as a causal structure embedded with human experience using structural causal models (SCMs). Within this framework, we rigorously define two core phenomena: unfaithful perception—where human interpretation diverges from the underlying causal mechanism—and inconsistent perception—where interpretations vary across individuals despite identical inputs. Innovatively, we model human experience as integrable causal priors, thereby extending the theoretical scope of SCMs to human-centered decision contexts. The framework unifies causal graph modeling, fairness analysis, and multi-decision-flow modeling to uncover how perceptual biases propagate into algorithmic unfairness. Empirical case studies demonstrate both the framework’s analytical validity and its practical applicability in real-world human-AI systems.

Technology Category

• Reasoning under Uncertainty: Causality
• Humans and AI: Learning Human Values and Preferences
• Machine Learning: Causal Learning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Perception occurs when two individuals interpret the same information differently. Despite being a known phenomenon with implications for bias in decision-making, as individual experience determines interpretation, perception remains largely overlooked in machine learning (ML) research. Modern decision flows, whether partially or fully automated, involve human experts interacting with ML applications. How might we then, e.g., account for two experts that interpret differently a deferred instance or an explanation from a ML model? To account for perception, we first need to formulate it. In this work, we define perception under causal reasoning using structural causal models (SCM). Our framework formalizes individual experience as additional causal knowledge that comes with and is used by a human expert (read, decision maker). We present two kinds of causal perception, unfaithful and inconsistent, based on the SCM properties of faithfulness and consistency. Further, we motivate the importance of perception within fairness problems. We illustrate our framework through a series of decision flow examples involving ML applications and human experts.

Problem

• Modeling perception differences in ML decision flows
• Integrating causal reasoning with expert interpretation variability
• Addressing perception's role in fairness of ML systems

Innovation

• Causal modeling of perception using SCM
• Defines structural and parametrical perception
• Integrates perception into ML decision flows