Learning from Similarity/Dissimilarity and Pairwise Comparison

2026-03-20

AI Summary

This work proposes SD-Pcomp, a novel framework for binary classification in the absence of instance-level explicit labels, leveraging only two types of weak supervision: pairwise class consistency (indicating whether two instances are similar or dissimilar) and positive-class preference (pairwise comparisons favoring the positive class). By constructing an unbiased risk estimator and employing a convex combination strategy to jointly model the relationship between these two forms of relative judgments, the method eliminates reliance on subjective probabilistic annotations. Both theoretical analysis and empirical evaluations demonstrate that SD-Pcomp significantly outperforms approaches using either weak signal alone across multiple datasets, while exhibiting strong robustness to label noise and errors in estimating the class prior.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

This paper addresses binary classification in scenarios where obtaining explicit instance level labels is impractical, by exploiting multiple weak labels defined on instance pairs. The existing SconfConfDiff classification framework relies on continuous valued probabilistic supervision, including similarity-confidence, the probability of class agreement, and confidence-difference, the difference in positive class probabilities. However, probabilistic labeling requires subjective uncertainty quantification, often leading to unstable supervision. We propose SD-Pcomp classification, a binary judgment based weakly supervised learning framework that relies only on relative judgments, namely class agreement between two instances and pairwise preference toward the positive class. The method employs Similarity/Dissimilarity (SD) labels and Pairwise Comparison (Pcomp) labels, and develops two unbiased risk estimators, (i) a convex combination of SD and Pcomp and (ii) a unified estimator that integrates both labels by modeling their relationship. Theoretical analysis and experimental results show that the proposed approach improves classification performance over methods using a single weak label, and is robust to label noise and uncertainty in class prior estimation.

Problem

• weakly supervised learning
• binary classification
• pairwise comparison
• similarity/dissimilarity
• label noise

Innovation

• SD-Pcomp
• weakly supervised learning
• pairwise comparison
• similarity/dissimilarity
• unbiased risk estimation