Transfer Learning in Regression with Influential Points

2025-09-24

AI Summary

In regression tasks where target-domain labels are scarce and influential points degrade transfer learning performance, this paper proposes a robust transfer learning co-optimization framework. The method uniquely integrates influential point detection with knowledge transfer via a joint source–target optimization objective, enabling simultaneous identification and suppression of influential points during parameter estimation. By unifying transfer learning, robust regression, and anomaly detection, the framework employs differentiable influence-weight learning for end-to-end co-optimization. Experiments on multiple synthetic and real-world cross-domain regression datasets demonstrate significant improvements: average MAE decreases by 12.7%, and influential point detection accuracy (F1-score) increases by 18.3%, outperforming state-of-the-art transfer and robust regression approaches.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Intelligent Robots: Learning & Optimization for ROB
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Regression prediction plays a crucial role in practical applications and strongly relies on data annotation. However, due to prohibitive annotation costs or domain-specific constraints, labeled data in the target domain is often scarce, making transfer learning a critical solution by leveraging knowledge from resource-rich source domains. In the practical target scenario, although transfer learning has been widely applied, influential points can significantly distort parameter estimation for the target domain model. This issue is further compounded when influential points are also present in source domains, leading to aggravated performance degradation and posing critical robustness challenges for existing transfer learning frameworks. In this study, we innovatively introduce a transfer learning collaborative optimization (Trans-CO) framework for influential point detection and regression model fitting. Extensive simulation experiments demonstrate that the proposed Trans-CO algorithm outperforms competing methods in terms of model fitting performance and influential point identification accuracy. Furthermore, it achieves superior predictive accuracy on real-world datasets, providing a novel solution for transfer learning in regression with influential points

Problem

• Detecting influential points in transfer learning regression
• Addressing performance degradation from source domain outliers
• Improving robustness of regression models across domains

Innovation

• Transfer learning collaborative optimization framework
• Detects influential points in source domains
• Improves regression model fitting accuracy