Ideological Bias in LLMs' Economic Causal Reasoning

2026-04-23

AI Summary

This study investigates whether large language models (LLMs) exhibit systematic ideological biases in economic causal reasoning, particularly in contexts where government intervention and market-oriented perspectives conflict. To this end, we extend the EconCausal benchmark by constructing a test set of 1,056 ideologically contested causal triplets grounded in empirical economics literature and evaluate the causal judgment accuracy of 20 prominent LLMs under zero-shot and one-shot prompting settings. Our analysis provides the first systematic identification and quantification of ideological bias in such tasks: 18 out of 20 models demonstrate higher accuracy when predictions align with interventionist expectations, and their erroneous predictions are significantly skewed toward interventionist outcomes. Notably, this bias persists even under one-shot prompting, indicating its robustness to minimal contextual guidance.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Game Theory and Economic Paradigms: Mechanism Design

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Do large language models (LLMs) exhibit systematic ideological bias when reasoning about economic causal effects? As LLMs are increasingly used in policy analysis and economic reporting, where directionally correct causal judgments are essential, this question has direct practical stakes. We present a systematic evaluation by extending the EconCausal benchmark with ideology-contested cases - instances where intervention-oriented (pro-government) and market-oriented (pro-market) perspectives predict divergent causal signs. From 10,490 causal triplets (treatment-outcome pairs with empirically verified effect directions) derived from top-tier economics and finance journals, we identify 1,056 ideology-contested instances and evaluate 20 state-of-the-art LLMs on their ability to predict empirically supported causal directions. We find that ideology-contested items are consistently harder than non-contested ones, and that across 18 of 20 models, accuracy is systematically higher when the empirically verified causal sign aligns with intervention-oriented expectations than with market-oriented ones. Moreover, when models err, their incorrect predictions disproportionately lean intervention-oriented, and this directional skew is not eliminated by one-shot in-context prompting. These results highlight that LLMs are not only less accurate on ideologically contested economic questions, but systematically less reliable in one ideological direction than the other, underscoring the need for direction-aware evaluation in high-stakes economic and policy settings.

Problem

• ideological bias
• economic causal reasoning
• large language models
• causal inference
• policy analysis

Innovation

• ideological bias
• causal reasoning
• large language models
• direction-aware evaluation
• EconCausal benchmark