Residual Semantic Decomposition of Word Embeddings

2026-05-17

AI Summary

Static word embeddings struggle to explicitly disambiguate polysemous words across different contexts. This work proposes Residual Semantic Decomposition (RSD), a method that recursively applies binary additive decomposition to incrementally extract semantic axes while analyzing residuals to uncover latent semantic information, all while preserving the original embedding space structure. Integrating neural additive modeling, contextual anchor diagnostics, and entropy analysis, RSD enables disentanglement of polysemous word meanings. Experiments demonstrate that RSD effectively isolates human-specified contextual semantic anchors, significantly outperforming random baselines. Furthermore, the study reveals that polysemous words in GloVe do not necessarily correspond to high-entropy boundary points, suggesting that residual neighborhoods are better suited for qualitative semantic diagnosis rather than standard sense prediction.

Abstract

We introduce Residual Semantic Decomposition (RSD), a neural additive decomposition of word embeddings that balances embedding reconstruction with relational structure preservation. RSD supports recursive binary decomposition: each $K=2$ fit extracts a local semantic axis, while residuals expose information not absorbed by that axis. In manually specified paired-context diagnostics over ambiguous words, RSD separates supplied context anchors above shuffled-label controls, but entropy diagnostics show that ambiguous targets are not uniformly high-entropy boundary points in static GloVe. We therefore treat residual neighborhoods as qualitative diagnostics rather than benchmark sense predictions.

Problem

• word embeddings
• semantic decomposition
• lexical ambiguity
• relational structure
• entropy diagnostics

Innovation

• Residual Semantic Decomposition
• word embeddings
• semantic axis
• recursive decomposition
• relational structure preservation