XTRA: Cross-Lingual Topic Modeling with Topic and Representation Alignments

2025-10-03

AI Summary

Cross-lingual topic modeling suffers from low topic coherence and inconsistent cross-lingual alignment. To address this, we propose DualAlign—a unified framework integrating bag-of-words representations with multilingual embeddings—featuring a dual-alignment mechanism: it jointly optimizes representation alignment (document/word embeddings) and topic alignment (document-topic and topic-word distributions) within a shared semantic space. DualAlign employs contrastive learning to enhance cross-lingual semantic consistency and introduces probabilistic distribution projection to synergistically optimize neural and traditional topic modeling objectives. Evaluated on multiple multilingual benchmark datasets, DualAlign achieves significant improvements over state-of-the-art baselines: +4.2% in topic coherence, +3.8% in topic diversity, and +6.1% in cross-lingual alignment quality. The method preserves interpretability inherent in probabilistic topic models while demonstrating strong generalization across languages and domains.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Multimodal Learning
• Computer Vision: Language and Vision

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Cross-lingual topic modeling aims to uncover shared semantic themes across languages. Several methods have been proposed to address this problem, leveraging both traditional and neural approaches. While previous methods have achieved some improvements in topic diversity, they often struggle to ensure high topic coherence and consistent alignment across languages. We propose XTRA (Cross-Lingual Topic Modeling with Topic and Representation Alignments), a novel framework that unifies Bag-of-Words modeling with multilingual embeddings. XTRA introduces two core components: (1) representation alignment, aligning document-topic distributions via contrastive learning in a shared semantic space; and (2) topic alignment, projecting topic-word distributions into the same space to enforce crosslingual consistency. This dual mechanism enables XTRA to learn topics that are interpretable (coherent and diverse) and well-aligned across languages. Experiments on multilingual corpora confirm that XTRA significantly outperforms strong baselines in topic coherence, diversity, and alignment quality. Code and reproducible scripts are available at https: //github.com/tienphat140205/XTRA.

Problem

• Uncover shared semantic themes across multiple languages
• Improve topic coherence and diversity in cross-lingual modeling
• Ensure consistent topic alignment across different languages

Innovation

• Unifies Bag-of-Words with multilingual embeddings
• Aligns document distributions via contrastive learning
• Projects topic-word distributions for crosslingual consistency