TopiCLEAR: Topic extraction by CLustering Embeddings with Adaptive dimensional Reduction

2025-12-07

AI Summary

To address semantic fragmentation, sparse word co-occurrence, and linguistic informality in social media short-text topic modeling, this paper proposes an end-to-end, preprocessing-free approach. It first generates sentence embeddings using Sentence-BERT, then applies supervised Linear Discriminant Analysis (LDA) for interpretable dimensionality reduction, and finally integrates a Gaussian Mixture Model (GMM) for iterative clustering optimization. The method operates directly on raw text, eliminating conventional cleaning and feature engineering steps. Experiments on four public short-text benchmarks demonstrate consistent superiority over seven baselines—including fine-tuned SBERT and zero-shot generative AI models—achieving an average 12.7% improvement in topic similarity. Notably, the gains are most pronounced on highly noisy, ultra-short social media content. The framework significantly enhances topic interpretability and alignment with human annotations, offering a robust, lightweight alternative for short-text thematic analysis.

Technology Category

• Natural Language Processing: Summarization
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Semantic Web

Application Category

• Web Mining and Content Analysis: Topic discovery and tracking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Rapid expansion of social media platforms such as X (formerly Twitter), Facebook, and Reddit has enabled large-scale analysis of public perceptions on diverse topics, including social issues, politics, natural disasters, and consumer sentiment. Topic modeling is a widely used approach for uncovering latent themes in text data, typically framed as an unsupervised classification task. However, traditional models, originally designed for longer and more formal documents, struggle with short social media posts due to limited co-occurrence statistics, fragmented semantics, inconsistent spelling, and informal language. To address these challenges, we propose a new method, TopiCLEAR: Topic extraction by CLustering Embeddings with Adaptive dimensional Reduction. Specifically, each text is embedded using Sentence-BERT (SBERT) and provisionally clustered using Gaussian Mixture Models (GMM). The clusters are then refined iteratively using a supervised projection based on linear discriminant analysis, followed by GMM-based clustering until convergence. Notably, our method operates directly on raw text, eliminating the need for preprocessing steps such as stop word removal. We evaluate our approach on four diverse datasets, 20News, AgNewsTitle, Reddit, and TweetTopic, each containing human-labeled topic information. Compared with seven baseline methods, including a recent SBERT-based method and a zero-shot generative AI method, our approach achieves the highest similarity to human-annotated topics, with significant improvements for both social media posts and online news articles. Additionally, qualitative analysis shows that our method produces more interpretable topics, highlighting its potential for applications in social media data and web content analytics.

Problem

• Extracts topics from short social media posts
• Handles informal language without preprocessing steps
• Improves topic similarity to human-labeled categories

Innovation

• Sentence-BERT embeddings for text representation
• Iterative clustering with adaptive linear discriminant analysis
• Direct raw text processing without preprocessing steps