🤖 AI Summary
Strategic analogical reasoning—particularly the “source-target matching” stage—has been overlooked as a distinct cognitive process, and the comparative capabilities of large language models (LLMs) versus humans remain poorly understood.
Method: We propose a causal-structure-mapping framework that moves beyond superficial similarity to rigorously evaluate analogical alignment. Using controlled behavioral experiments and causal alignment assessments, we compare GPT-4 and human performance in strategic analogy generation and evaluation.
Contribution/Results: GPT-4 exhibits high recall but low precision—prone to surface-level matches—whereas humans show the inverse pattern. Their error profiles are complementary: LLMs lack causal modeling capacity, while humans often misinterpret underlying mechanisms. Building on this, we introduce a novel human-AI collaboration paradigm wherein AI generates candidate analogies and humans perform causal validation. Empirical results demonstrate that this division of cognitive labor significantly improves strategic analogy quality, offering a practical, cognitively grounded pathway for AI-augmented organizational decision-making.
📝 Abstract
This study investigates whether large language models, specifically GPT4, can match human capabilities in analogical reasoning within strategic decision making contexts. Using a novel experimental design involving source to target matching, we find that GPT4 achieves high recall by retrieving all plausible analogies but suffers from low precision, frequently applying incorrect analogies based on superficial similarities. In contrast, human participants exhibit high precision but low recall, selecting fewer analogies yet with stronger causal alignment. These findings advance theory by identifying matching, the evaluative phase of analogical reasoning, as a distinct step that requires accurate causal mapping beyond simple retrieval. While current LLMs are proficient in generating candidate analogies, humans maintain a comparative advantage in recognizing deep structural similarities across domains. Error analysis reveals that AI errors arise from surface level matching, whereas human errors stem from misinterpretations of causal structure. Taken together, the results suggest a productive division of labor in AI assisted organizational decision making where LLMs may serve as broad analogy generators, while humans act as critical evaluators, applying the most contextually appropriate analogies to strategic problems.