The BD-LSC Dataset: Facilitating the Benchmarking of Models for Lexical Semantic Change Detection in Slang and Standard Usage

2026-06-15

AI Summary

This study addresses the challenge of modeling bidirectional semantic change—such as concurrent sense gain and loss—in contexts where slang coexists with standard usage. To this end, the authors introduce two complementary benchmark datasets: BD-LSC for cross-temporal semantic change analysis and ST-WSD for fine-grained word sense disambiguation, both featuring instance-level sense annotations that explicitly capture sense acquisition, loss, and stability. A comprehensive evaluation employing contextual embedding clustering, supervised learning, Transformer-based models, and large language models—including GPT-4o—reveals that GPT-4o achieves the highest performance in Exact Sense Match and multi-label accuracy. Nevertheless, all models attain only a Macro-F1 of approximately 0.5 on rare slang senses, underscoring the persistent difficulty of this task.

Abstract

Automatic semantic change detection aims to identify how word meanings shift over time, offering insights into both linguistic and societal change. Despite recent progress in computational lexical semantic change (LSC), existing benchmarks and methods struggle to capture bi-directional semantic change, particularly cases where words simultaneously gain and lose senses. This problem is especially challenging for words that have both slang and standard meanings. To address these gaps, we introduce two complementary benchmark datasets. The Bi-Directional Lexical Semantic Change (BD-LSC) dataset captures sense gain, sense loss, and stability across three time periods, enabling the study of complex semantic trajectories. The SlangTrack Word Sense Disambiguation (ST-WSD) dataset provides fine-grained, instance-level sense annotations for words combining slang and standard usages, supporting systematic benchmarking of WSD and semantic change detection models. Using these benchmarks, we systematically evaluate models across different methodological families: unsupervised clustering using contextualised embeddings, supervised machine learning, transformer-based models, and state-of-the-art large language models. Among the evaluated systems, the few-shot GPT-4o model achieved the strongest aggregate performance on Exact Sense Match (ESM) and multi-label accuracy; however, Macro-F1 scores near 0.5 across all systems show that rare slang senses remain difficult, which we identify as the central open challenge.

Problem

• lexical semantic change
• bi-directional semantic change
• slang
• sense gain and loss
• word sense disambiguation

Innovation

• lexical semantic change
• bi-directional semantic shift
• slang detection
• word sense disambiguation
• benchmark dataset