Neural Topic Modeling with Large Language Models in the Loop

2024-11-13

arXiv.org

AI Summary

To address incomplete topic coverage, semantic misalignment, and low inference efficiency when directly applying large language models (LLMs) to topic modeling, this paper proposes an LLM-in-the-loop collaborative framework that synergistically integrates LLMs with neural topic models (NTMs). Our core contribution is a dynamic confidence-driven topic alignment mechanism grounded in optimal transport, which complements LLMs’ strong semantic understanding with NTMs’ efficient latent-variable representation—without requiring LLM fine-tuning and maintaining compatibility with mainstream NTM architectures. The framework preserves document representation fidelity while substantially enhancing topic interpretability and coherence. Extensive experiments on multiple benchmark datasets demonstrate consistent superiority over both existing LLM-augmented and purely neural topic modeling approaches, exhibiting strong generalization and controllable computational overhead.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Topic modeling is a fundamental task in natural language processing, allowing the discovery of latent thematic structures in text corpora. While Large Language Models (LLMs) have demonstrated promising capabilities in topic discovery, their direct application to topic modeling suffers from issues such as incomplete topic coverage, misalignment of topics, and inefficiency. To address these limitations, we propose LLM-ITL, a novel LLM-in-the-loop framework that integrates LLMs with Neural Topic Models (NTMs). In LLM-ITL, global topics and document representations are learned through the NTM. Meanwhile, an LLM refines these topics using an Optimal Transport (OT)-based alignment objective, where the refinement is dynamically adjusted based on the LLM's confidence in suggesting topical words for each set of input words. With the flexibility of being integrated into many existing NTMs, the proposed approach enhances the interpretability of topics while preserving the efficiency of NTMs in learning topics and document representations. Extensive experiments demonstrate that LLM-ITL helps NTMs significantly improve their topic interpretability while maintaining the quality of document representation. Our code and datasets are available at https://github.com/Xiaohao-Yang/LLM-ITL

Problem

• Enhances topic interpretability in neural topic models
• Improves alignment and coverage of discovered topics
• Combines LLMs with NTMs for efficient topic modeling

Innovation

• Integrates LLMs with Neural Topic Models
• Uses Optimal Transport for topic alignment
• Dynamically adjusts refinement based on LLM confidence