Exploring Concreteness Through a Figurative Lens

2026-04-20

AI Summary

Static concreteness ratings fail to capture the dynamic shifts in word concreteness within figurative contexts, and it remains unclear how large language models (LLMs) represent such variation. This study systematically investigates how four classes of LLMs distinguish between literal and figurative uses of nouns and organize concreteness in their representational spaces through inter-layer geometric analysis. The findings reveal that models separate literal from figurative usages as early as the initial layers, and compress concreteness into a consistent one-dimensional direction across models in intermediate to later layers. Leveraging this direction, the work achieves, for the first time, concreteness-guided text generation without fine-tuning and demonstrates its effectiveness in metaphor classification and style-controllable rewriting.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Symbolic Representations

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Static concreteness ratings are widely used in NLP, yet a word's concreteness can shift with context, especially in figurative language such as metaphor, where common concrete nouns can take abstract interpretations. While such shifts are evident from context, it remains unclear how LLMs understand concreteness internally. We conduct a layer-wise and geometric analysis of LLM hidden representations across four model families, examining how models distinguish literal vs figurative uses of the same noun and how concreteness is organized in representation space. We find that LLMs separate literal and figurative usage in early layers, and that mid-to-late layers compress concreteness into a one-dimensional direction that is consistent across models. Finally, we show that this geometric structure is practically useful: a single concreteness direction supports efficient figurative-language classification and enables training-free steering of generation toward more literal or more figurative rewrites.

Problem

• concreteness
• figurative language
• metaphor
• large language models
• contextual meaning

Innovation

• concreteness
• figurative language
• geometric analysis
• representation space
• LLM interpretability