Contrastive Learning for Semi-Supervised Deep Regression with Generalized Ordinal Rankings from Spectral Seriation

2025-12-09

AI Summary

To address the strong reliance on expensive labeled data and insufficient exploitation of unlabeled data in semi-supervised deep regression, this paper proposes a novel framework integrating spectral seriation and contrastive learning. First, a feature-similarity graph is constructed jointly from labeled and unlabeled samples; spectral seriation is then applied to provably recover reliable ordinal relationships among unlabeled instances. Second, a label-guided robust ordinal regularization mechanism is designed, coupled with a dynamic-programming-driven feature selection strategy to enhance representation discriminability. The method unifies pseudo-label supervision, ordinal modeling, and contrastive learning within a single coherent framework. Extensive experiments demonstrate significant improvements over state-of-the-art semi-supervised regression methods across multiple benchmark datasets. Theoretical analysis guarantees consistent ordinal relationship recovery. Code is publicly available.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Learning to Search
• Computer Vision: Representation Learning for Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Contrastive learning methods enforce label distance relationships in feature space to improve representation capability for regression models. However, these methods highly depend on label information to correctly recover ordinal relationships of features, limiting their applications to semi-supervised regression. In this work, we extend contrastive regression methods to allow unlabeled data to be used in the semi-supervised setting, thereby reducing the dependence on costly annotations. Particularly we construct the feature similarity matrix with both labeled and unlabeled samples in a mini-batch to reflect inter-sample relationships, and an accurate ordinal ranking of involved unlabeled samples can be recovered through spectral seriation algorithms if the level of error is within certain bounds. The introduction of labeled samples above provides regularization of the ordinal ranking with guidance from the ground-truth label information, making the ranking more reliable. To reduce feature perturbations, we further utilize the dynamic programming algorithm to select robust features for the matrix construction. The recovered ordinal relationship is then used for contrastive learning on unlabeled samples, and we thus allow more data to be used for feature representation learning, thereby achieving more robust results. The ordinal rankings can also be used to supervise predictions on unlabeled samples, serving as an additional training signal. We provide theoretical guarantees and empirical verification through experiments on various datasets, demonstrating that our method can surpass existing state-of-the-art semi-supervised deep regression methods. Our code have been released on https://github.com/xmed-lab/CLSS.

Problem

• Extends contrastive regression to semi-supervised settings using unlabeled data
• Recovers ordinal rankings from unlabeled samples via spectral seriation algorithms
• Reduces dependence on costly annotations for robust regression models

Innovation

• Contrastive learning with spectral seriation for ordinal ranking
• Dynamic programming selects robust features for similarity matrix
• Unlabeled data improves regression via recovered ordinal relationships