Towards regularized learning from functional data with covariate shift

2026-01-28

AI Summary

This work addresses unsupervised domain adaptation for functional-output regression under covariate shift by proposing a regularized operator learning framework grounded in vector-valued reproducing kernel Hilbert spaces (vRKHS). The approach ensures stable estimation of functional outputs through constrained hypothesis spaces and establishes minimax optimal convergence rates under source conditioning assumptions. A novel, parameter-free ensemble strategy is introduced to automatically aggregate estimators derived from diverse regularization parameters and kernel functions, thereby enhancing generalization without manual tuning. Empirical validation on real-world facial image datasets demonstrates the method’s robustness and effectiveness, with both theoretical analysis and experimental results confirming its significant mitigation of performance degradation caused by distributional shift.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Learning & Optimization for CV
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

This paper investigates a general regularization framework for unsupervised domain adaptation in vector-valued regression under the covariate shift assumption, utilizing vector-valued reproducing kernel Hilbert spaces (vRKHS). Covariate shift occurs when the input distributions of the training and test data differ, introducing significant challenges for reliable learning. By restricting the hypothesis space, we develop a practical operator learning algorithm capable of handling functional outputs. We establish optimal convergence rates for the proposed framework under a general source condition, providing a theoretical foundation for regularized learning in this setting. We also propose an aggregation-based approach that forms a linear combination of estimators corresponding to different regularization parameters and different kernels. The proposed approach addresses the challenge of selecting appropriate tuning parameters, which is crucial for constructing a good estimator, and we provide a theoretical justification for its effectiveness. Furthermore, we illustrate the proposed method on a real-world face image dataset, demonstrating robustness and effectiveness in mitigating distributional discrepancies under covariate shift.

Problem

• covariate shift
• unsupervised domain adaptation
• vector-valued regression
• functional data
• regularized learning

Innovation

• regularized learning
• covariate shift
• vector-valued RKHS
• operator learning
• aggregation-based estimator