Near-optimal Rank Adaptive Inference of High Dimensional Matrices

2025-10-09

AI Summary

This paper addresses adaptive estimation of high-dimensional matrices from linear measurements. To overcome the limitations of conventional fixed-rank assumptions, we propose a data-driven rank-adaptive inference framework that iteratively estimates singular values and vectors to dynamically determine the effective rank, balancing singular value accuracy against low-rank approximation error. Theoretically, we derive instance-specific lower bounds on sample complexity, characterizing the fundamental trade-off in effective rank selection. Algorithmically, our method integrates least-squares estimation with universal singular value thresholding, leveraging enhanced matrix denoising analysis to achieve near-optimal recovery. We establish rigorous finite-sample error bounds and empirically validate the approach on multivariate regression and linear system identification tasks, demonstrating performance that approaches theoretical limits.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Intelligent Robots: State Estimation
• Search and Optimization: Non-convex Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

We address the problem of estimating a high-dimensional matrix from linear measurements, with a focus on designing optimal rank-adaptive algorithms. These algorithms infer the matrix by estimating its singular values and the corresponding singular vectors up to an effective rank, adaptively determined based on the data. We establish instance-specific lower bounds for the sample complexity of such algorithms, uncovering fundamental trade-offs in selecting the effective rank: balancing the precision of estimating a subset of singular values against the approximation cost incurred for the remaining ones. Our analysis identifies how the optimal effective rank depends on the matrix being estimated, the sample size, and the noise level. We propose an algorithm that combines a Least-Squares estimator with a universal singular value thresholding procedure. We provide finite-sample error bounds for this algorithm and demonstrate that its performance nearly matches the derived fundamental limits. Our results rely on an enhanced analysis of matrix denoising methods based on singular value thresholding. We validate our findings with applications to multivariate regression and linear dynamical system identification.

Problem

• Estimating high-dimensional matrices from linear measurements
• Designing optimal rank-adaptive algorithms for matrix inference
• Balancing precision and approximation costs in rank selection

Innovation

• Adaptive rank determination from data measurements
• Least-Squares combined with singular value thresholding
• Near-optimal finite-sample error bounds performance