Transfer Learning for Moderate-Dimensional Ridge-Regularized Robust Linear Regression

2026-03-31

AI Summary

This work proposes Trans-RR, a transfer learning method for robust regression in the moderate-dimensional setting where the number of predictors is of the same order as the sample size and regression coefficients are non-sparse. Trans-RR integrates a robust ridge estimator from the source domain with a robust ridge correction from the target domain to achieve effective knowledge transfer. It establishes the first provably effective transfer learning framework for non-sparse moderate-dimensional robust regression, providing theoretical characterization of estimation error and uncovering both positive and negative transfer phenomena. The analysis demonstrates that incorporating source data can substantially improve estimation accuracy. Numerical simulations and real-data experiments corroborate the method’s efficacy and its sensitivity to distributional discrepancies between source and target domains.

Abstract

This paper studies transfer learning for ridge-regularized robust linear regression in the moderate-dimensional regime, where the number of predictors is of the same order as the sample size and the regression coefficients are not assumed to be sparse. We propose Trans-RR, which combines a robust ridge estimator from a source study with a robust ridge correction based on the target study. Under mild assumptions, we characterize the asymptotic estimation error of the proposed estimator and show that leveraging source data can substantially improve estimation accuracy relative to the traditional single-study ridge-regularized robust estimator. Simulation results and a real-data analysis support the theory and illustrate both positive and negative transfer as the discrepancy between the source and target studies varies.

Problem

• transfer learning
• ridge regression
• robust linear regression
• moderate-dimensional regime
• estimation accuracy

Innovation

• transfer learning
• ridge regression
• robust estimation
• moderate-dimensional regime
• asymptotic analysis