AI-enhanced semantic feature norms for 786 concepts

2025-05-15

AI Summary

Traditional semantic feature norming research faces a trade-off between conceptual coverage breadth and annotation quality due to prohibitive human labor costs. Method: We propose an LLM-augmented paradigm to construct NOVA—a high-density semantic feature dataset covering 786 concepts—by integrating human-elicited features with LLM-generated ones, followed by rigorous expert validation and behavioral experiments. Contribution/Results: This work achieves the first credible, human-verified integration of LLM-generated features with canonical human norms. We demonstrate that human conceptual knowledge substantially exceeds existing norming datasets, yielding significantly higher feature density and inter-concept overlap. In predicting human semantic similarity judgments, NOVA consistently outperforms both pure human norming datasets and state-of-the-art word embedding models (e.g., BERT, GloVe). Our approach establishes a novel AI-augmented paradigm for cognitive science data construction, balancing scalability, fidelity, and empirical validity.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Application Domains: Humanities & Computational Social Science
• Humans and AI: Crowd Sourcing and Human Computation

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Semantic feature norms have been foundational in the study of human conceptual knowledge, yet traditional methods face trade-offs between concept/feature coverage and verifiability of quality due to the labor-intensive nature of norming studies. Here, we introduce a novel approach that augments a dataset of human-generated feature norms with responses from large language models (LLMs) while verifying the quality of norms against reliable human judgments. We find that our AI-enhanced feature norm dataset, NOVA: Norms Optimized Via AI, shows much higher feature density and overlap among concepts while outperforming a comparable human-only norm dataset and word-embedding models in predicting people's semantic similarity judgments. Taken together, we demonstrate that human conceptual knowledge is richer than captured in previous norm datasets and show that, with proper validation, LLMs can serve as powerful tools for cognitive science research.

Problem

• Enhancing semantic feature norms using AI for broader concept coverage
• Validating AI-generated norms against human judgments for quality assurance
• Demonstrating AI-enhanced norms outperform human-only datasets in semantic tasks

Innovation

• AI-enhanced semantic feature norms for concepts
• Combines human-generated norms with LLM responses
• Validates norms against human judgments