Low-Rank Plus Sparse Matrix Transfer Learning under Growing Representations and Ambient Dimensions

2026-01-29

AI Summary

This work addresses the challenge of structured matrix transfer learning under joint growth of environmental dimensions and intrinsic representations. The authors propose a general framework that decomposes target parameters into an embedded source-task component, a low-rank increment, and a sparse correction. By leveraging an anchored alternating projection algorithm, the method estimates only the low-dimensional novel components while preserving the transferred subspace. This approach achieves structured transfer for the first time in settings where both dimensionality and representation evolve jointly. The derived error bound cleanly separates contributions from target noise, representation expansion, and source estimation error, yielding significantly improved convergence rates under low-rank and sparsity assumptions on the increments. Theoretical analysis provides end-to-end guarantees for single-trajectory estimation of Markov transition matrices, and covariance estimation experiments empirically validate consistent transfer gains, corroborating the method’s efficacy both theoretically and experimentally.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Search and Optimization: Learning to Search
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Learning systems often expand their ambient features or latent representations over time, embedding earlier representations into larger spaces with limited new latent structure. We study transfer learning for structured matrix estimation under simultaneous growth of the ambient dimension and the intrinsic representation, where a well-estimated source task is embedded as a subspace of a higher-dimensional target task. We propose a general transfer framework in which the target parameter decomposes into an embedded source component, low-dimensional low-rank innovations, and sparse edits, and develop an anchored alternating projection estimator that preserves transferred subspaces while estimating only low-dimensional innovations and sparse modifications. We establish deterministic error bounds that separate target noise, representation growth, and source estimation error, yielding strictly improved rates when rank and sparsity increments are small. We demonstrate the generality of the framework by applying it to two canonical problems. For Markov transition matrix estimation from a single trajectory, we derive end-to-end theoretical guarantees under dependent noise. For structured covariance estimation under enlarged dimensions, we provide complementary theoretical analysis in the appendix and empirically validate consistent transfer gains.

Problem

• transfer learning
• low-rank plus sparse
• growing representations
• ambient dimensions
• structured matrix estimation

Innovation

• low-rank plus sparse decomposition
• transfer learning
• growing representations
• anchored alternating projection
• structured matrix estimation