Shuffled Linear Regression via Spectral Matching

2024-09-30

arXiv.org

AI Summary

This paper addresses large-scale linear regression under unknown permutations of measurement dimensions—i.e., observations are linearly mapped to a high-dimensional latent feature space after an unknown reordering. Conventional least squares (LS) and LASSO estimators fail in this setting due to permutation ambiguity. To overcome the combinatorial intractability of direct permutation search, we propose a novel spectral matching method based on covariance spectrum alignment, enabling joint estimation of the linear transformation and the underlying permutation structure without explicit combinatorial optimization. We establish theoretical guarantees showing that our estimator consistently recovers the shuffled LS/LASSO solutions asymptotically. Furthermore, we extend the framework to image registration, jointly estimating pose parameters and keypoint correspondences within a unified formulation. Extensive experiments on synthetic data and real-world image registration tasks demonstrate that our approach achieves significantly higher estimation accuracy and registration performance compared to state-of-the-art methods.

Technology Category

• Machine Learning: Learning with Manifolds
• Intelligent Robots: State Estimation
• Search and Optimization: Learning to Search

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Shuffled linear regression (SLR) seeks to estimate latent features through a linear transformation, complicated by unknown permutations in the measurement dimensions. This problem extends traditional least-squares (LS) and Least Absolute Shrinkage and Selection Operator (LASSO) approaches by jointly estimating the permutation, resulting in shuffled LS and shuffled LASSO formulations. Existing methods, constrained by the combinatorial complexity of permutation recovery, often address small-scale cases with limited measurements. In contrast, we focus on large-scale SLR, particularly suited for environments with abundant measurement samples. We propose a spectral matching method that efficiently resolves permutations by aligning spectral components of the measurement and feature covariances. Rigorous theoretical analyses demonstrate that our method achieves accurate estimates in both shuffled LS and shuffled LASSO settings, given a sufficient number of samples. Furthermore, we extend our approach to address simultaneous pose and correspondence estimation in image registration tasks. Experiments on synthetic datasets and real-world image registration scenarios show that our method outperforms existing algorithms in both estimation accuracy and registration performance.

Problem

• Estimating latent features with unknown permutations in measurements
• Extending LS and LASSO by jointly estimating permutations
• Solving large-scale SLR efficiently via spectral matching

Innovation

• Spectral matching resolves permutation efficiently
• Handles large-scale shuffled linear regression
• Extends to image registration tasks