Unsupervised Contrastive Learning Using Out-Of-Distribution Data for Long-Tailed Dataset

2025-06-15

AI Summary

To address the representation imbalance and weak discriminability of self-supervised learning (SSL) under long-tailed data distributions, this paper pioneers the integration of out-of-distribution (OOD) unlabeled data into long-tailed SSL frameworks. We propose a pseudo-semantic discrimination loss that jointly optimizes semantic alignment and distribution alignment via a domain discrimination auxiliary task. Additionally, we design a guidance-driven contrastive learning mechanism, wherein a lightweight guidance network dynamically modulates positive/negative sample selection and adjusts attraction/repulsion strengths in the embedding space. Our approach synergistically combines OOD data sampling, dual-task joint optimization, and embedding-space transfer—requiring no human annotation. Extensive experiments on four benchmark long-tailed image datasets demonstrate significant improvements over state-of-the-art methods: average tail-class classification accuracy increases by 4.2%, and representation separability—measured by inter-class distance over intra-class compactness—improves by 18.7%.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Learning to Search
• Computer Vision: Representation Learning for Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Representation, semantic annotation, enhancement, enrichments, access and/or integration of a variety of data on the Web
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

This work addresses the task of self-supervised learning (SSL) on a long-tailed dataset that aims to learn balanced and well-separated representations for downstream tasks such as image classification. This task is crucial because the real world contains numerous object categories, and their distributions are inherently imbalanced. Towards robust SSL on a class-imbalanced dataset, we investigate leveraging a network trained using unlabeled out-of-distribution (OOD) data that are prevalently available online. We first train a network using both in-domain (ID) and sampled OOD data by back-propagating the proposed pseudo semantic discrimination loss alongside a domain discrimination loss. The OOD data sampling and loss functions are designed to learn a balanced and well-separated embedding space. Subsequently, we further optimize the network on ID data by unsupervised contrastive learning while using the previously trained network as a guiding network. The guiding network is utilized to select positive/negative samples and to control the strengths of attractive/repulsive forces in contrastive learning. We also distil and transfer its embedding space to the training network to maintain balancedness and separability. Through experiments on four publicly available long-tailed datasets, we demonstrate that the proposed method outperforms previous state-of-the-art methods.

Problem

• Learning balanced representations for long-tailed datasets using SSL
• Leveraging unlabeled OOD data to improve class-imbalanced learning
• Enhancing contrastive learning with a guiding network for separability

Innovation

• Leveraging unlabeled OOD data for SSL
• Pseudo semantic and domain discrimination losses
• Guiding network for contrastive learning optimization