On the Eligibility of LLMs for Counterfactual Reasoning: A Decompositional Study

2025-05-17

AI Summary

Large language models (LLMs) exhibit inconsistent performance in counterfactual reasoning, and the underlying causes—particularly the role of modality dependence—remain poorly understood. Method: We propose the first decomposable evaluation framework for counterfactual reasoning, disentangling the task into two sequential stages: *causal structure construction* and *counterfactual intervention reasoning*. We conduct systematic benchmarking across 11 cross-modal datasets spanning text, mathematics, code, and vision-language domains. Contribution/Results: Through stage-wise behavioral analysis, we uncover— for the first time—the critical influence of modality type and intermediate reasoning steps on LLMs’ counterfactual capabilities. We precisely identify *causal modeling* (not intervention reasoning) as the primary bottleneck. Our framework establishes an interpretable diagnostic pathway and provides both theoretical grounding and concrete optimization directions for enhancing LLMs’ robust counterfactual reasoning.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Reasoning under Uncertainty: Causality
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Counterfactual reasoning has emerged as a crucial technique for generalizing the reasoning capabilities of large language models (LLMs). By generating and analyzing counterfactual scenarios, researchers can assess the adaptability and reliability of model decision-making. Although prior work has shown that LLMs often struggle with counterfactual reasoning, it remains unclear which factors most significantly impede their performance across different tasks and modalities. In this paper, we propose a decompositional strategy that breaks down the counterfactual generation from causality construction to the reasoning over counterfactual interventions. To support decompositional analysis, we investigate 11 datasets spanning diverse tasks, including natural language understanding, mathematics, programming, and vision-language tasks. Through extensive evaluations, we characterize LLM behavior across each decompositional stage and identify how modality type and intermediate reasoning influence performance. By establishing a structured framework for analyzing counterfactual reasoning, this work contributes to the development of more reliable LLM-based reasoning systems and informs future elicitation strategies.

Problem

• Identify factors impeding LLMs' counterfactual reasoning performance
• Decompose counterfactual reasoning into causality and intervention stages
• Evaluate LLMs across 11 diverse tasks and modalities

Innovation

• Decompositional strategy for counterfactual reasoning stages
• Evaluation across 11 diverse task datasets
• Structured framework analyzing modality and reasoning impact