Nearly Linear Sparsification of $ell_p$ Subspace Approximation

2024-07-03

AI Summary

This paper studies coreset construction for the $k$-dimensional subspace approximation problem under the $ell_p$ norm. For the NP-hard offline and online settings, we propose the first nearly optimal-size strong coresets: of size $ ilde{O}(k)cdotmathrm{poly}(varepsilon^{-1})$ for $p<2$, and $ ilde{O}(k^{p/2})cdotmathrm{poly}(varepsilon^{-1})$ for $p>2$ in the offline case; and—crucially—the first online strong coreset achieving the same asymptotic size bounds. Our approach integrates $ell_p$-sensitivity analysis, importance sampling, and streaming compression techniques, rigorously preserving the geometric structure of the original point set. Compared to prior work, we eliminate all redundant $mathrm{poly}(k)$ factors, achieving—under a $(1+varepsilon)$-relative error guarantee—the theoretically optimal coreset size for both offline and online settings.

Technology Category

• Machine Learning: Learning with Manifolds
• Data Mining & Knowledge Management: Data Compression
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements

Abstract

The $ell_p$ subspace approximation problem is an NP-hard low rank approximation problem that generalizes the median hyperplane problem ($p = 1$), principal component analysis ($p = 2$), and the center hyperplane problem ($p = infty$). A popular approach to cope with the NP-hardness of this problem is to compute a strong coreset, which is a small weighted subset of the input points which simultaneously approximates the cost of every $k$-dimensional subspace, typically to $(1+varepsilon)$ relative error for a small constant $varepsilon$. We obtain the first algorithm for constructing a strong coreset for $ell_p$ subspace approximation with a nearly optimal dependence on the rank parameter $k$, obtaining a nearly linear bound of $ ilde O(k)mathrm{poly}(varepsilon^{-1})$ for $p<2$ and $ ilde O(k^{p/2})mathrm{poly}(varepsilon^{-1})$ for $p>2$. Prior constructions either achieved a similar size bound but produced a coreset with a modification of the original points [SW18, FKW21], or produced a coreset of the original points but lost $mathrm{poly}(k)$ factors in the coreset size [HV20, WY23]. Our techniques also lead to the first nearly optimal online strong coresets for $ell_p$ subspace approximation with similar bounds as the offline setting, resolving a problem of [WY23]. All prior approaches lose $mathrm{poly}(k)$ factors in this setting, even when allowed to modify the original points.

Problem

• Core-Set
• p-Norm Subspace Approximation
• Online Algorithms

Innovation

• p-norm subspace approximation
• streaming optimal core-sets
• k-parameter minimization