Inverse Entropic Optimal Transport Solves Semi-supervised Learning via Data Likelihood Maximization

2024-10-03

arXiv.org

AI Summary

This work addresses the estimation of the conditional distribution π*(·|x) in semi-supervised learning, proposing an end-to-end method that jointly leverages a small set of labeled (paired) data and abundant unlabeled marginal samples (x ∼ π*_x, y ∼ π*_y). Methodologically, it formulates the problem via Inverse Entropic Optimal Transport (IEOT), unifying paired and unpaired learning under a principled maximum-data-likelihood objective—eliminating heuristic design choices. The IEOT framework enables seamless co-optimization of both data modalities, with efficient computation via the Sinkhorn algorithm. Theoretically, this is the first work to establish a rigorous connection between semi-supervised learning and IEOT. Empirically, the approach achieves state-of-the-art performance on domain adaptation tasks, significantly outperforming conventional semi-supervised baselines with minimal computational overhead—demonstrating both theoretical soundness and practical efficiency.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Learning to Search
• Computer Vision: Learning & Optimization for CV

Application Category

• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Learning conditional distributions $pi^*(cdot|x)$ is a central problem in machine learning, which is typically approached via supervised methods with paired data $(x,y) sim pi^*$. However, acquiring paired data samples is often challenging, especially in problems such as domain translation. This necessitates the development of $ extit{semi-supervised}$ models that utilize both limited paired data and additional unpaired i.i.d. samples $x sim pi^*_x$ and $y sim pi^*_y$ from the marginal distributions. The usage of such combined data is complex and often relies on heuristic approaches. To tackle this issue, we propose a new learning paradigm that integrates both paired and unpaired data $ extbf{seamlessly}$ through the data likelihood maximization techniques. We demonstrate that our approach also connects intriguingly with inverse entropic optimal transport (OT). This finding allows us to apply recent advances in computational OT to establish a $ extbf{light}$ learning algorithm to get $pi^*(cdot|x)$. Furthermore, we demonstrate through empirical tests that our method effectively learns conditional distributions using paired and unpaired data simultaneously.

Problem

• Semi-supervised Learning
• Data Integration
• Conditional Data Distribution

Innovation

• Reverse Entropy Optimal Transport
• Semi-supervised Learning
• Condition Distribution Learning