SMART: Semantic Matching Contrastive Learning for Partially View-Aligned Clustering

2025-12-17

AI Summary

In partially aligned multi-view clustering (PVC), semantic matching of unaligned samples remains challenging, and cross-view representation distribution shifts lead to inaccurate correspondence estimation. To address these issues, this paper proposes a novel framework integrating semantic matching with contrastive learning. We introduce the first cross-view distribution correction mechanism to jointly model shared semantic structures for both aligned and unaligned samples. Additionally, we incorporate self-supervised feature disentanglement and multi-view consistency regularization to mitigate representation shifts caused by view heterogeneity. Evaluated on eight benchmark datasets, our method achieves an average 4.2% improvement in clustering accuracy over state-of-the-art approaches, demonstrating superior robustness—especially under low alignment ratios. Our core contributions are: (1) a semantic-matching-driven contrastive learning framework; (2) a cross-view distribution alignment mechanism; and (3) a unified semantic structure modeling paradigm tailored for partial alignment settings.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Multi-modal Vision
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Multi-view clustering has been empirically shown to improve learning performance by leveraging the inherent complementary information across multiple views of data. However, in real-world scenarios, collecting strictly aligned views is challenging, and learning from both aligned and unaligned data becomes a more practical solution. Partially View-aligned Clustering aims to learn correspondences between misaligned view samples to better exploit the potential consistency and complementarity across views, including both aligned and unaligned data. However, most existing PVC methods fail to leverage unaligned data to capture the shared semantics among samples from the same cluster. Moreover, the inherent heterogeneity of multi-view data induces distributional shifts in representations, leading to inaccuracies in establishing meaningful correspondences between cross-view latent features and, consequently, impairing learning effectiveness. To address these challenges, we propose a Semantic MAtching contRasTive learning model (SMART) for PVC. The main idea of our approach is to alleviate the influence of cross-view distributional shifts, thereby facilitating semantic matching contrastive learning to fully exploit semantic relationships in both aligned and unaligned data. Extensive experiments on eight benchmark datasets demonstrate that our method consistently outperforms existing approaches on the PVC problem.

Problem

• Addresses partially view-aligned clustering with misaligned data
• Mitigates cross-view distributional shifts for semantic matching
• Exploits semantic relationships in both aligned and unaligned data

Innovation

• Semantic matching contrastive learning for clustering
• Alleviates cross-view distributional shifts
• Exploits semantic relationships in aligned and unaligned data