As Easy as Rocket Science: Assessing the Ability of Large Language Models to Interpret Negation in Figurative Language

2026-06-17

AI Summary

This study addresses the underexplored challenge of large language models (LLMs) in comprehending linguistic expressions that simultaneously involve negation and figurative language—a common yet complex phenomenon in everyday discourse. The work presents the first systematic effort to incorporate negation into the evaluation of figurative language understanding by manually annotating existing metaphor datasets with negated variants and assessing a range of state-of-the-art LLMs under diverse prompting strategies. Results demonstrate that the presence of negation substantially impairs model performance on figurative interpretation, with accuracy highly sensitive to prompt formulation and varying markedly across different types of negation. These findings underscore a critical limitation in current LLMs’ capacity to handle intricate semantic compositionality involving both figurative meaning and logical operators such as negation.

Abstract

Figurative language and negation are two areas that challenge current language models, however, both are widely used throughout written and spoken language. Large language models (LLMs) are also widely used in everyday contexts where they cannot necessarily be tuned for a specific dataset. It is therefore essential to understand the ability of LLMs to correctly interpret text that includes both negation and figurative language. To investigate this, we develop a set of new annotations to an existing dataset of figurative language, and test a range of language models on the dataset. We find that the combination of negation and figurativeness can present a particular challenge, and that performance overall and across different negation types is particularly dependent on the prompt style used.

Problem

• negation
• figurative language
• large language models
• language understanding
• natural language processing

Innovation

• negation
• figurative language
• large language models
• prompt sensitivity
• annotation dataset