Misalignment of Semantic Relation Knowledge between WordNet and Human Intuition

2024-12-03

arXiv.org

AI Summary

This study investigates the alignment between WordNet’s expert-curated semantic relations (e.g., synonymy, hypernymy) and human intuitive judgments. Using large-scale human-subject experiments grounded in semantic templates—combined with rigorous statistical analysis and qualitative pattern mining—we empirically demonstrate that WordNet’s path-length metric fails to capture human intuitions of hypernymic distance, and that systematic mismatches exist in both synonymy and taxonomic structure. These findings expose a fundamental tension between lexicographic knowledge and lay cognition, challenging the implicit cognitive assumptions underlying prevailing semantic resources. Methodologically, the work introduces quantifiable cognitive validity metrics for dictionary evaluation. Substantively, it provides the first empirical foundation for developing cognitively aligned semantic resources—bridging computational lexicons and human conceptual organization. (138 words)

Technology Category

• Natural Language Processing: Lexical Semantics and Morphology
• Data Mining & Knowledge Management: Semantic Web
• Cognitive Modeling & Cognitive Systems: Analogy

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

WordNet provides a carefully constructed repository of semantic relations, created by specialists. But there is another source of information on semantic relations, the intuition of language users. We present the first systematic study of the degree to which these two sources are aligned. Investigating the cases of misalignment could make proper use of WordNet and facilitate its improvement. Our analysis which uses templates to elicit responses from human participants, reveals a general misalignment of semantic relation knowledge between WordNet and human intuition. Further analyses find a systematic pattern of mismatch among synonymy and taxonomic relations~(hypernymy and hyponymy), together with the fact that WordNet path length does not serve as a reliable indicator of human intuition regarding hypernymy or hyponymy relations.

Problem

• Studies misalignment between WordNet and human semantic intuition
• Examines mismatch in synonymy and taxonomic relations
• Assesses WordNet path length unreliability for human intuition

Innovation

• Systematic study of WordNet-human intuition alignment
• Templates elicit human responses for comparison
• Analyze synonymy and taxonomic relation mismatches