Fast Dimensionality Reduction from $ell_2$ to $ell_p$

2025-10-29

AI Summary

This paper addresses the problem of fast linear dimensionality reduction from Euclidean space $ell_2^d$ to $ell_p^k$ for $p in [1,2]$, aiming for $varepsilon$-distance preservation with low distortion. The proposed method generalizes the Ailon–Liberty framework, yielding the first construction that extends optimal $ell_2 o ell_1$ embeddings to arbitrary $p in [1,2]$. It combines randomized projections with fast Hadamard transforms, achieving an embedding time complexity of $O(d log k)$—significantly improving upon prior approaches, especially for small $k$. Furthermore, the paper establishes a dimension lower bound applicable to all target $ell_p$ norms, which tightly matches the known optimal lower bound. Both theoretical analysis and empirical evaluation confirm that the method achieves optimal asymptotic trade-offs in both computational efficiency and distance preservation.

Technology Category

• Machine Learning: Dimensionality Reduction/Feature Selection
• Data Mining & Knowledge Management: Data Compression
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

The Johnson-Lindenstrauss (JL) lemma is a fundamental result in dimensionality reduction, ensuring that any finite set $X subseteq mathbb{R}^d$ can be embedded into a lower-dimensional space $mathbb{R}^k$ while approximately preserving all pairwise Euclidean distances. In recent years, embeddings that preserve Euclidean distances when measured via the $ell_1$ norm in the target space have received increasing attention due to their relevance in applications such as nearest neighbor search in high dimensions. A recent breakthrough by Dirksen, Mendelson, and Stollenwerk established an optimal $ell_2 o ell_1$ embedding with computational complexity $O(d log d)$. In this work, we generalize this direction and propose a simple linear embedding from $ell_2$ to $ell_p$ for any $p in [1,2]$ based on a construction of Ailon and Liberty. Our method achieves a reduced runtime of $O(d log k)$ when $k leq d^{1/4}$, improving upon prior runtime results when the target dimension is small. Additionally, we show that for emph{any norm} $|cdot|$ in the target space, any embedding of $(mathbb{R}^d, |cdot|_2)$ into $(mathbb{R}^k, |cdot|)$ with distortion $varepsilon$ generally requires $k = Ωig(varepsilon^{-2} log(varepsilon^2 n)/log(1/varepsilon)ig)$, matching the optimal bound for the $ell_2$ case up to a logarithmic factor.

Problem

• Generalizing dimensionality reduction from Euclidean to p-norm spaces
• Improving computational efficiency for small target dimensions
• Establishing lower bounds for distortion in norm embeddings

Innovation

• Linear embedding from l2 to lp using Ailon-Liberty construction
• Achieves reduced runtime O(d log k) for small k
• Provides optimal dimension bound for any norm embedding