Is analogy enough to draw novel adjective-noun inferences?

2025-03-31

AI Summary

This study investigates whether human and large language model (LLM) interpretation of novel adjective–noun compounds relies solely on analogical reasoning or requires deeper compositional semantic mechanisms. Method: We construct a lexical-similarity–based analogical model, conduct controlled human behavioral experiments, and systematically evaluate both humans and LLMs on the Ross et al. (2025) benchmark dataset for novel compound interpretation. Contribution/Results: Our analysis provides the first empirical characterization of analogical reasoning’s capacity limits in this task. While analogy successfully explains many frequent, semantically transparent compounds, it fails on several semantically novel ones—yet humans and LLMs maintain high agreement and plausibility in their interpretations. This consistency reveals reliance on compositionally grounded semantic mechanisms beyond analogy. The findings furnish critical behavioral evidence for linguistic compositionality, challenge purely analogical accounts of novel compound interpretation, and affirm the necessity of structured compositional semantics in human and artificial language understanding.

Technology Category

• Cognitive Modeling & Cognitive Systems: Analogy
• Natural Language Processing: Lexical Semantics and Morphology
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Recent work (Ross et al., 2025, 2024) has argued that the ability of humans and LLMs respectively to generalize to novel adjective-noun combinations shows that they each have access to a compositional mechanism to determine the phrase's meaning and derive inferences. We study whether these inferences can instead be derived by analogy to known inferences, without need for composition. We investigate this by (1) building a model of analogical reasoning using similarity over lexical items, and (2) asking human participants to reason by analogy. While we find that this strategy works well for a large proportion of the dataset of Ross et al. (2025), there are novel combinations for which both humans and LLMs derive convergent inferences but which are not well handled by analogy. We thus conclude that the mechanism humans and LLMs use to generalize in these cases cannot be fully reduced to analogy, and likely involves composition.

Problem

• Can analogy alone explain novel adjective-noun inferences?
• Comparing analogical vs. compositional mechanisms in humans and LLMs
• Identifying cases where analogy fails to match human/LLM generalization

Innovation

• Modeling analogical reasoning with lexical similarity
• Testing human analogy-based inference performance
• Identifying limitations of analogy in novel combinations