Beyond Top Words: MonoTM for Topic Modeling with Interpretable Monosemantic Features

2026-09-08

AI Summary

该研究提出MonoTM框架，通过稀疏自动编码器提取语义特征，以改善主题模型的可解释性和质量。

Abstract

Topic models summarize large text corpora, but top-ranked words often provide only a limited representation of topic semantics. Sparse autoencoders (SAEs) offer a way to move beyond word-level descriptors by extracting interpretable features from dense representations, yet how feature interpretability relates to topic-inference quality remains unclear. We introduce \textbf{MonoTM}, an interpretable topic modeling framework that decouples these roles. Across three benchmark corpora, we show that document--topic mixture estimation and semantic interpretation favor different SAE configurations and feature subsets. MonoTM estimates mixtures from the full SAE bag-of-features representation and, with them fixed, learns topic descriptors over a separate vocabulary of corpus-grounded semantic features. This design preserves global topic structure while representing topics with semantic units more meaningful than individual words, making them more useful for downstream corpus analysis.

Problem

• topic modeling
• interpretable features
• sparse autoencoders
• semantic interpretation

Innovation

• Sparse Autoencoders
• Interpretable Features
• Topic Modeling
• MonoTM
• Semantics