Transparent Semantic Spaces: A Categorical Approach to Explainable Word Embeddings

2025-08-28

AI Summary

This work addresses three key challenges in word embedding research: weak interpretability of embedding models, the absence of a unified comparative framework across algorithms, and the difficulty of quantifying semantic bias. To this end, we propose a transparent semantic space framework grounded in category theory. Our method formalizes textual semantics via categorical structures—specifically, the Conf (configuration) and Emb (embedding) categories—introduces divergence decoration, and defines dimension-agnostic semantic spaces, enabling mathematically rigorous modeling and visualization of semantic extraction. We establish, for the first time under strict mathematical conditions, the categorical equivalence of GloVe, Word2Vec, and multidimensional scaling (MDS) within a shared structural framework. Moreover, the framework supports computable characterization of pre-embedding bias and bias mitigation at the semantic level. By transforming black-box embeddings into verifiable, comparable, and controllable systems, our approach provides a formal foundation for interpretable AI and fair semantic modeling.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Transparent, Interpretable, Explainable ML

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

The paper introduces a novel framework based on category theory to enhance the explainability of artificial intelligence systems, particularly focusing on word embeddings. Key topics include the construction of categories $ Ł_{T} $ and $ ¶_{T} $, providing schematic representations of the semantics of a text $ T $, and reframing the selection of the element with maximum probability as a categorical notion. Additionally, the monoidal category $ ¶_{T} $ is constructed to visualize various methods of extracting semantic information from $ T $, offering a dimension-agnostic definition of semantic spaces reliant solely on information within the text. Furthermore, the paper defines the categories of configurations $ Conf $ and word embeddings $ Emb $, accompanied by the concept of divergence as a decoration on $ Emb $. It establishes a mathematically precise method for comparing word embeddings, demonstrating the equivalence between the GloVe and Word2Vec algorithms and the metric MDS algorithm, transitioning from neural network algorithms (black box) to a transparent framework. Finally, the paper presents a mathematical approach to computing biases before embedding and offers insights on mitigating biases at the semantic space level, advancing the field of explainable artificial intelligence.

Problem

• Enhancing explainability of AI systems using category theory
• Providing transparent framework for comparing word embeddings
• Computing and mitigating biases at semantic space level

Innovation

• Category theory framework for explainable word embeddings
• Monoidal category construction for dimension-agnostic semantic spaces
• Mathematical equivalence between neural and transparent algorithms