Approximating Uniform Random Rotations by Two-Block Structured Hadamard Rotations in High Dimensions

2026-04-25

AI Summary

This work investigates the approximation capability of a structured random rotation—composed of two Walsh–Hadamard transforms interleaved with a random diagonal sign matrix—relative to a uniform random rotation in high dimensions. While such structured rotations are widely adopted in practice due to their computational efficiency, their theoretical guarantees have remained elusive. Leveraging tools from Kolmogorov and Wasserstein distances, high-dimensional geometry, and random matrix theory, the paper establishes the first unified convergence bound for one-dimensional marginals, showing an approximation error of $O(d^{-1/5})$. Moreover, it constructs tight upper and lower bounds on the full-dimensional Wasserstein distance in worst-case scenarios, revealing fundamental limitations in globally approximating uniform random rotations in high dimensions and thereby delineating the precise regime where this structured alternative remains effective.

Technology Category

• Machine Learning: Learning with Manifolds
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Bridging structured and unstructured data

Abstract

Uniform random rotations are a useful primitive in applications such as fast Johnson-Lindenstrauss embeddings, kernel approximation, communication-efficient learning, and recent AI compression pipelines, but they are computationally expensive to generate and apply in high dimensions. A common practical replacement is repeated structured random rotations built from Walsh-Hadamard transforms and random sign diagonals. Applying the structured random rotation twice has been shown empirically to be useful, but the supporting theory is still limited. In this paper we study the approximation quality achieved when using this two-block structured Hadamard rotation. Our results are both positive and negative. On the positive side, we prove that every fixed coordinate of the two-block transform converges uniformly, over all inputs, to the corresponding coordinate of a uniformly rotated vector, with an explicit Kolmogorov-distance bound of order $d^{-1/5}$. On the negative side, we prove an explicit lower bound on the Wasserstein distance between the full vector distributions, showing that the two-block transform is not a globally accurate surrogate for a uniform random rotation in the worst case. For the extremal input used in the lower bound, we also prove a matching asymptotic upper bound, showing that the lower-bound scale is sharp for that input. Taken together, the results identify a clear separation between one-dimensional marginal behavior, where approximation improves with dimension, and full high-dimensional geometry, where a nonvanishing discrepancy remains. This provides a partial theoretical explanation for the empirical success of structured Hadamard rotations in some algorithms, while also clarifying the limitations of treating them as drop-in replacements for true uniform random rotations.

Problem

• uniform random rotations
• structured Hadamard rotations
• high-dimensional approximation
• Johnson-Lindenstrauss embeddings
• Wasserstein distance

Innovation

• structured random rotations
• Hadamard transform
• uniform random rotations
• high-dimensional approximation
• distributional convergence