Optimal Transport for Domain Adaptation through Gaussian Mixture Models

2024-03-18

arXiv.org

AI Summary

To address distribution shift and computational scalability challenges in unsupervised domain adaptation, this paper proposes an optimal transport (OT) method formulated directly in the parameter space of Gaussian mixture models (GMMs). Unlike conventional sample-level Wasserstein OT—whose complexity is cubic in sample size—our approach is the first to model OT at the GMM parameter level, yielding a closed-form transport mapping with approximate time complexity of *O(K²d)*, where *K* is the number of mixture components and *d* the feature dimension. By operating on distribution parameters rather than raw samples, the method avoids explicit sample dependence, enabling efficient, scalable distribution alignment in high-dimensional and large-scale settings. It offers strong theoretical interpretability and integrates naturally into shallow-domain-adaptation frameworks. Extensive evaluation across 85 cross-domain tasks on nine benchmark datasets demonstrates consistent and significant improvements over state-of-the-art shallow adaptation methods.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Machine learning systems operate under the assumption that training and test data are sampled from a fixed probability distribution. However, this assumptions is rarely verified in practice, as the conditions upon which data was acquired are likely to change. In this context, the adaptation of the unsupervised domain requires minimal access to the data of the new conditions for learning models robust to changes in the data distribution. Optimal transport is a theoretically grounded tool for analyzing changes in distribution, especially as it allows the mapping between domains. However, these methods are usually computationally expensive as their complexity scales cubically with the number of samples. In this work, we explore optimal transport between Gaussian Mixture Models (GMMs), which is conveniently written in terms of the components of source and target GMMs. We experiment with 9 benchmarks, with a total of $85$ adaptation tasks, showing that our methods are more efficient than previous shallow domain adaptation methods, and they scale well with number of samples $n$ and dimensions $d$.

Problem

• Adaptive Machine Learning
• Minimal Data Requirement
• Performance Maintenance

Innovation

• Gaussian Mixture Models
• Optimal Transport Theory
• Data Domain Adaptation