Quantifying Lexical Semantic Shift via Unbalanced Optimal Transport

2024-12-17

arXiv.org

AI Summary

This paper addresses the challenge of identifying and quantifying instance-level semantic change in lexical semantic change detection. We propose a contextualized word embedding alignment framework based on Unbalanced Optimal Transport (UOT). Our key contributions are threefold: (1) We are the first to apply UOT to lexical semantic change modeling, enabling fine-grained, usage-level characterization of semantic shift; (2) We introduce the Sense Usage Shift (SUS) metric, which jointly supports instance-level change quantification, word-level change magnitude estimation, and discrimination between semantic broadening and narrowing; (3) Our method achieves state-of-the-art performance across multiple benchmark datasets, significantly improving accuracy in instance-level change detection while providing interpretable, usage-level semantic shift visualizations.

Technology Category

• Natural Language Processing: Lexical Semantics and Morphology
• Data Mining & Knowledge Management: Semantic Web
• Knowledge Representation and Reasoning: Action, Change, and Causality

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Sentiment analysis and opinion mining

Abstract

Lexical semantic change detection aims to identify shifts in word meanings over time. While existing methods using embeddings from a diachronic corpus pair estimate the degree of change for target words, they offer limited insight into changes at the level of individual usage instances. To address this, we apply Unbalanced Optimal Transport (UOT) to sets of contextualized word embeddings, capturing semantic change through the excess and deficit in the alignment between usage instances. In particular, we propose Sense Usage Shift (SUS), a measure that quantifies changes in the usage frequency of a word sense at each usage instance. By leveraging SUS, we demonstrate that several challenges in semantic change detection can be addressed in a unified manner, including quantifying instance-level semantic change and word-level tasks such as measuring the magnitude of semantic change and the broadening or narrowing of meaning.

Problem

• Detects lexical semantic shifts in word meanings over time
• Measures instance-level semantic change using contextualized embeddings
• Quantifies usage frequency changes of word senses per instance

Innovation

• UOT for contextualized embeddings alignment
• Sense Usage Shift quantifies sense frequency
• Unified solution for semantic change tasks