Taming the Randomness: Towards Label-Preserving Cropping in Contrastive Learning

2025-04-28

AI Summary

In contrastive learning, random cropping often induces semantic distortion and spurious label assignments, degrading representation quality. To address this, we propose a label-preserving parametric cropping strategy, introducing two novel learnable cropping mechanisms: (i) geometrically aligned cropping via differentiable spatial transformations, and (ii) region-aware cropping guided by semantic saliency—both ensuring semantic consistency between cropped views and the original image, thereby mitigating false negative samples and erroneous self-supervised signals at their source. Our method is fully compatible with standard contrastive frameworks (e.g., SimCLR) and supports end-to-end training. On CIFAR-10 downstream classification, it outperforms random cropping by 2.7–12.4% in accuracy, demonstrating substantial improvements in model generalization and representation discriminability.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Contrastive learning (CL) approaches have gained great recognition as a very successful subset of self-supervised learning (SSL) methods. SSL enables learning from unlabeled data, a crucial step in the advancement of deep learning, particularly in computer vision (CV), given the plethora of unlabeled image data. CL works by comparing different random augmentations (e.g., different crops) of the same image, thus achieving self-labeling. Nevertheless, randomly augmenting images and especially random cropping can result in an image that is semantically very distant from the original and therefore leads to false labeling, hence undermining the efficacy of the methods. In this research, two novel parameterized cropping methods are introduced that increase the robustness of self-labeling and consequently increase the efficacy. The results show that the use of these methods significantly improves the accuracy of the model by between 2.7% and 12.4% on the downstream task of classifying CIFAR-10, depending on the crop size compared to that of the non-parameterized random cropping method.

Problem

• Addresses false labeling in contrastive learning due to random cropping
• Introduces parameterized cropping to enhance self-labeling robustness
• Improves model accuracy by 2.7%-12.4% on CIFAR-10 classification

Innovation

• Introduces parameterized cropping for contrastive learning
• Enhances self-labeling robustness in image augmentation
• Improves model accuracy significantly on CIFAR-10