Tracing the ongoing emergence of human-like reasoning in Large Language Models

2026-05-20

AI Summary

This study investigates whether large language models (LLMs) possess human-like pragmatic reasoning abilities, with a focus on the comprehension of conditional statements. Through psycholinguistic experiments across four languages, it presents the first systematic comparison of 25 LLMs against matched human participants on identical conditional reasoning tasks. The findings reveal that, despite demonstrating accurate semantic processing, LLMs consistently lack the pragmatic inference characteristic of human cognition. This deficiency persists regardless of model architecture, training objective, or open-source status, underscoring pragmatic reasoning as a fundamental limitation in current artificial systems. The work makes a novel contribution by establishing a multilingual, multi-model benchmark and advancing methods for cognitive alignment evaluation in natural language processing.

Abstract

Humans effortlessly go beyond literal meanings: If you mow the lawn, I will give you fifty dollars, is typically understood as implying that the speaker will pay only if the lawn is mowed, whereas If you are hungry, there is pizza in the oven implies that pizza is available regardless of the hearers hunger. Large Language Models - LLMs - show human-like performance on many tasks, yet it remains unclear whether they reason like humans. To address this, we conducted a population-matching experiment assessing how twentyfive LLMs compute conditional inferences across four languages, compared to an equal number of humans per language. We find that humans enrich logical reasoning through pragmatic inferences across languages. Model behavior is more variable. Some LLMs perfectly follow the truth-table of conditionals but they ignore pragmatic inferences, while others deviate from the truth-table, adhering to a single interpretation across the board, thus reflecting accurate rule-based processing but not human-like reasoning. Overall, LLMs are accurate semantic operators, but fail to capture the pragmatic enrichments characteristic of human reasoning. Crucially, LLM accuracy is neither predicted nor boosted by open vs. closed status, training orientation, or architecture type, suggesting that pragmatic reasoning is still an emerging ability in the cognitive toolkit of artificial systems.

Problem

• pragmatic reasoning
• conditional inference
• Large Language Models
• human-like reasoning
• truth-table

Innovation

• pragmatic reasoning
• conditional inference
• large language models
• cross-lingual evaluation
• human-like reasoning