Transfer Learning for Linear Discriminant Analysis with a Shared Classification Signal

2026-07-07

AI Summary

This work addresses multi-source domain transfer learning for high-dimensional binary classification by proposing a linear discriminant analysis framework that jointly models shared discriminative signals and domain-specific biases. Under both homogeneous and heterogeneous covariance assumptions, the authors leverage spiked covariance models to conduct high-dimensional asymptotic analysis, deriving deterministic limits for the target-domain classification error. This yields an asymptotically optimal oracle transfer weighting scheme, along with a consistent plug-in estimator, and provides an optimal intercept correction under class imbalance. The theoretical and methodological developments rigorously quantify how transfer performance is influenced by the strength of shared signals, domain heterogeneity, the dimension-to-sample ratio, and the spiked structure of the covariance matrices, thereby achieving asymptotically optimal classification with controllable error.

Abstract

This paper studies transfer learning for linear discriminant analysis in high-dimensional two-class classification. We consider one target domain and several source domains, where the mean difference in each domain is decomposed into a deterministic common component and a domain-specific random deviation. The common component represents a shared classification signal across domains, while the random deviation captures domain-specific heterogeneity. Under spiked covariance models, we derive deterministic limits for the target-domain Gaussian-calibrated error of weighted transfer classifiers under both homogeneous and heterogeneous covariance settings. These limits quantify the effects of the shared signal, domain-specific variation, dimension-to-sample-size ratios, and spike structures on transfer performance. They further lead to oracle transfer weights and consistent data-driven plug-in estimators. We also characterize the intercept bias induced by unbalanced target-domain class sample sizes and provide an asymptotically optimal correction.

Problem

• transfer learning
• linear discriminant analysis
• shared classification signal
• high-dimensional classification
• domain heterogeneity

Innovation

• transfer learning
• linear discriminant analysis
• shared classification signal
• spiked covariance model
• high-dimensional asymptotics