leverage score estimation

Computing exact leverage scores for Kronecker-product structured matrices in large-scale least-squares problems is computationally prohibitive, while existing approximation methods incur statistical bias and high overhead. Method: We propose the first efficient exact leverage score algorithm tailored to Kronecker-structured matrices. Leveraging the inherent tensor structure, our method designs a near-linear-time framework for exact leverage score computation and sampling—bypassing costly full-matrix SVD or biased sketching approximations. Contribution/Results: Theoretically and empirically, our algorithm achieves significantly lower sampling error than state-of-the-art approximate methods (e.g., FJLT- or CountSketch-accelerated approaches), while maintaining substantially lower time complexity than full SVD. This work establishes the first scalable, exact, and efficient leverage score sampling scheme for Kronecker-structured matrices, enabling improved structured random projections and large-scale regression.

This project addresses efficient randomized algorithms for three fundamental matrix computations under resource constraints: (1) low-rank positive semidefinite (PSD) matrix approximation, aiming for exact low-rank reconstruction with minimal element accesses; (2) estimation of implicit matrix properties—trace, diagonal entries, and row norms—using only matrix-vector products; and (3) numerically robust floating-point solutions to overdetermined linear least-squares problems. Methodologically, we propose a randomized pivoted Cholesky decomposition to drastically reduce element accesses in PSD approximation; introduce a novel leave-one-out framework for property estimation, achieving optimal sample complexity; and design the first randomized least-squares solver that simultaneously attains asymptotic acceleration—O(nd log d) time for n × d matrices—and numerical backward stability, matching the accuracy of direct methods. These contributions advance both theoretical guarantees and practical efficiency in large-scale, memory- or query-constrained numerical linear algebra.

Structured random matrices lack a unified theoretical analysis and a general design framework in randomized linear algebra. Method: This paper introduces the “Oblivious Subspace Injection” (OSI) property, establishing the first decoupled abstract analytical framework that separates correctness proofs of algorithms from instantiation-specific verification. Contribution/Results: We prove that sparse random matrices, random triangular transforms, and tensor-product-structured matrices all satisfy OSI, thereby unifying their dimensionality-reduction fidelity guarantees for tasks such as low-rank approximation and least-squares regression. Leveraging this framework, we design accelerated algorithms with near-optimal time complexity. Empirical evaluation on synthetic datasets and scientific computing benchmarks confirms both efficiency and practical utility.

Weighted Low-Rank Approximation (WLRA) seeks a rank-$k$ matrix $XY^ op$ minimizing the weighted Frobenius norm $|W circ (M - XY^ op)|_F$, given a matrix $M$ and a nonnegative weight matrix $W$. This problem is NP-hard and hard to approximate. This paper proposes the first alternating minimization framework for WLRA that simultaneously achieves strong theoretical guarantees and high efficiency. Our method integrates a high-accuracy multi-response regression solver into each alternating update step, enabling approximate yet controllable subproblem solving. Crucially, it preserves global convergence while reducing the per-iteration time complexity from $O(|W|_0 k^2)$ to $O(|W|_0 k)$, where $|W|_0$ denotes the number of nonzero entries in $W$—yielding substantial speedups for sparse weighting patterns. Experiments demonstrate state-of-the-art performance on matrix completion and noise-robust recovery tasks.

This work addresses efficiency bottlenecks in solving large-scale linear systems and approximating matrix norms. We propose a multilevel randomized sketching preconditioned iterative method, integrating Nyström low-rank approximation, sparse random sketching, and multilevel preconditioning. It establishes the first multilevel sketched preconditioning framework grounded in the natural average condition number. Theoretical contributions include: (1) optimal complexity $ ilde{O}(n^2 + d_lambda^omega)$ for solving regularized linear systems; (2) accelerated complexity $ ilde{O}(n^{2.065} + k^omega)$ for systems with $k$ outlying singular values; and (3) Schatten-$p$ norm approximation—particularly the nuclear norm—at $ ilde{O}(n^{2.11})$, improving upon the prior best $ ilde{O}(n^{2.18})$. These advances significantly enhance computational efficiency for key subproblems in applications such as Gaussian process regression.

This work addresses the problem of efficiently and accurately computing ℓ_p-Lewis weights (for p ≥ 4) of a matrix, which quantify the importance of its rows. By alternating between primal and dual formulations of the underlying optimization problem and integrating leverage score iteration with a locally relative smooth gradient descent method, the authors propose a novel algorithm that significantly reduces computational overhead while maintaining high accuracy. Specifically, the proposed approach improves the iteration complexity from O(p³ log(m/ε)) to O(p² log(m/ε)), achieving the current best-known bound on the number of iterations required for ε-approximate Lewis weight computation.

This work addresses the grouped distributionally robust (GDR) least squares problem, which seeks to minimize the worst-case loss across multiple data groups. The authors introduce block Lewis weights—a novel geometric tool—to reformulate the problem as a specially weighted least squares instance. By integrating an accelerated proximal algorithm with a structured linear system solver tailored for systems of the form \(A^\top B A\), they achieve an efficient solution method that unifies optimization frameworks for both average and robust losses. The proposed approach outperforms interior-point methods at moderate accuracy levels. Theoretically, it attains a \((1+\varepsilon)\)-approximate solution using only \(\widetilde{O}(\min\{\mathrm{rank}(A), m\}^{1/3} \varepsilon^{-2/3})\) linear system solves, yielding the current best-known guarantee for the special case of \(\ell_\infty\) regression.

This work addresses the high computational cost of traditional leverage-score-based algorithms for approximating the John ellipsoid to within a $(1+\varepsilon)$ factor, whose iteration complexity scales with $\varepsilon^{-1}$ and thus hinders high-precision efficiency. By reframing the problem as D-optimal design, the paper reveals that the $\varepsilon^{-1}$ dependence stems solely from conventional certification mechanisms. It proposes a novel strategy that focuses exclusively on the final iteration and avoids solution averaging. Leveraging leverage scores as a first-order gradient oracle, the method combines warm-start acceleration with a damped Newton algorithm exploiting the self-concordance of the barrier function to accurately recover the Hessian. This approach achieves a $(1+\varepsilon)$-approximation using only $O(d^2 \log\log(1/\varepsilon))$ oracle queries and incurs a preprocessing cost $C(A)$ independent of $\varepsilon$, substantially reducing the overhead for high-accuracy computation.

This work addresses the high computational cost of ordinary least squares (OLS) estimation in large-scale linear models by proposing the Sequentially Preconditioned Conjugate Gradient (SPCG) method. SPCG constructs a sequence of increasingly larger randomized sketch subproblems, each solved via an inner preconditioned conjugate gradient (PCG) iteration, and leverages warm-starting from the previous solution to accelerate convergence. The final solution is refined on the full-scale problem. SPCG uniquely integrates incremental sketching with warm-started PCG, significantly reducing both iteration count and CPU time while preserving OLS prediction accuracy. Theoretical analysis provides rigorous guarantees on convergence and computational complexity, and empirical results demonstrate its superiority over full-data PCG and the Iterative Double Sketching (IDS) method.

This work investigates whether the linear dependence on the restricted condition number κ in sparse least-squares optimization can be overcome in polynomial time. Under the randomized exact Small-Set Expansion hypothesis, the authors establish—via a complexity-theoretic reduction coupled with a refined analysis of the restricted condition number—the first proof of the Axiotis–Sviridenko conjecture in this setting: for any fixed γ ∈ (0,1], no polynomial-time randomized algorithm can output, with probability exceeding 2/3, a solution of sparsity O(k·κ^{1−γ}) that meets the target error, even when the input instance consists of rational entries and has full column rank. A key component of the argument was derived with the aid of an AI-driven automated theorem-proving system, thereby establishing a theoretical lower bound on the condition number barrier.