SuperCM: Improving Semi-Supervised Learning and Domain Adaptation through differentiable clustering

2025-07-18

AI Summary

In semi-supervised learning (SSL) and unsupervised domain adaptation (UDA), the scarcity of labeled data leads to suboptimal utilization of unlabeled samples. Method: This paper introduces an explicit, differentiable clustering module that end-to-end integrates the clustering assumption into model training: it explicitly optimizes cluster centroids using few labeled samples and enforces unlabeled samples to follow class-consistent soft cluster assignments in feature space. Contribution/Results: The module is plug-and-play—serving either as a generic regularizer to enhance existing SSL/UDA methods or as the core of a lightweight standalone training framework. Evaluated on multiple SSL and UDA benchmarks, our approach significantly outperforms state-of-the-art methods under extremely low labeling ratios (e.g., 1% labeled data), demonstrating that explicit differentiable clustering effectively improves model generalization and class discriminability.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Learning to Search
• Computer Vision: Diffusion Models for Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Semi-Supervised Learning (SSL) and Unsupervised Domain Adaptation (UDA) enhance the model performance by exploiting information from labeled and unlabeled data. The clustering assumption has proven advantageous for learning with limited supervision and states that data points belonging to the same cluster in a high-dimensional space should be assigned to the same category. Recent works have utilized different training mechanisms to implicitly enforce this assumption for the SSL and UDA. In this work, we take a different approach by explicitly involving a differentiable clustering module which is extended to leverage the supervised data to compute its centroids. We demonstrate the effectiveness of our straightforward end-to-end training strategy for SSL and UDA over extensive experiments and highlight its benefits, especially in low supervision regimes, both as a standalone model and as a regularizer for existing approaches.

Problem

• Improves SSL and UDA via differentiable clustering
• Explicitly enforces clustering assumption with supervised data
• Enhances performance in low supervision regimes

Innovation

• Differentiable clustering module for SSL and UDA
• Leveraging supervised data to compute centroids
• End-to-end training strategy for low supervision