Unified Optimization of Source Weights and Transfer Quantities in Multi-Source Transfer Learning: An Asymptotic Framework

2026-01-15

AI Summary

This work addresses negative transfer in multi-source transfer learning, which often arises from heterogeneity among source tasks. Existing approaches typically optimize either source weights or the number of transferred samples in isolation, neglecting their joint optimization. The authors formulate this challenge as a parameter estimation problem based on KL divergence–derived generalization error and propose a unified asymptotic analysis framework to simultaneously optimize both source weights and sample sizes. Theoretically, they establish—for the first time—that after proper weight adaptation, utilizing all available source samples is optimal, and they provide a closed-form solution for the single-source case along with a convex optimization algorithm for the multi-source setting. Extensive experiments on benchmarks such as DomainNet and Office-Home demonstrate significant improvements over strong baselines, corroborating both the theoretical predictions and practical efficacy of the proposed method.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Transfer learning plays a vital role in improving model performance in data-scarce scenarios. However, naive uniform transfer from multiple source tasks may result in negative transfer, highlighting the need to properly balance the contributions of heterogeneous sources. Moreover, existing transfer learning methods typically focus on optimizing either the source weights or the amount of transferred samples, while largely neglecting the joint consideration of the other. In this work, we propose a theoretical framework, Unified Optimization of Weights and Quantities (UOWQ), which formulates multi-source transfer learning as a parameter estimation problem grounded in an asymptotic analysis of a Kullback-Leibler divergence-based generalization error measure. The proposed framework jointly determines the optimal source weights and optimal transfer quantities for each source task. Firstly, we prove that using all available source samples is always optimal once the weights are properly adjusted, and we provide a theoretical explanation for this phenomenon. Moreover, to determine the optimal transfer weights, our analysis yields closed-form solutions in the single-source setting and develops a convex optimization-based numerical procedure for the multi-source case. Building on the theoretical results, we further propose practical algorithms for both multi-source transfer learning and multi-task learning settings. Extensive experiments on real-world benchmarks, including DomainNet and Office-Home, demonstrate that UOWQ consistently outperforms strong baselines. The results validate both the theoretical predictions and the practical effectiveness of our framework.

Problem

• multi-source transfer learning
• negative transfer
• source weights
• transfer quantities
• heterogeneous sources

Innovation

• multi-source transfer learning
• unified optimization
• asymptotic analysis
• Kullback-Leibler divergence
• convex optimization