Improving TensorSketch Using Complex Random Variables

2026-08-11

AI Summary

This work addresses the issue of exponentially growing variance in traditional TensorSketch when estimating high-order polynomial kernels, where the variance scales as $3^p/D$ with the degree $p$, severely degrading accuracy. The authors propose a novel TensorSketch variant based on complex-valued random variables, introducing complex random projections into the sparse framework of Pham et al. for the first time. This approach maintains the original time complexity of $O(p(\text{nnz}(x) + D \log D))$ while significantly reducing the variance bound to $2^p/D$. Both theoretical analysis and empirical evaluations demonstrate that the proposed method consistently improves estimation accuracy and computational efficiency across synthetic and real-world datasets.

Abstract

\texttt{TensorSketch} by~\cite{pham2013fast,kar2012random} provides efficient sketching algorithms for high-dimensional polynomial kernels $\vec{x}^{\otimes p} \in \R^{d^p}$. \cite{kar2012random} uses dense Johnson-Lindenstrauss (JL)-type projections with computational cost $O(pDd)$, where $D$ denotes the sketch dimension, whereas~\cite{pham2013fast} extends the sparse \texttt{CountSketch}~\citep{count_sketch} algorithm, yielding a faster algorithm for high-dimensional sparse inputs with running time $O\big(p(\nnz{\vec{x}} + D \log D)\big)$. However, the variance of both estimators grows exponentially with the polynomial degree $p$, scaling as $3^{p}/D$. Recent work by~\cite{pmlr-v206-wacker23a} showed that using complex-valued distribution reduces this dependence to $2^{p}/D$ for the approach of~\cite{kar2012random}. However, their method relies on dense JL-type projections with computational cost $O(pDd)$ and does not extend to the algorithm of~\cite{pham2013fast}. In this work, we introduce a simple variant of \texttt{TensorSketch}~\citep{pham2013fast} that achieves the same variance bound as~\cite{pmlr-v206-wacker23a}, while retaining its advantage of the input-sparsity running time. We validate our results with supporting experiments on synthetic and real-world datasets.

Problem

• TensorSketch
• polynomial kernels
• variance reduction
• complex random variables
• input sparsity

Innovation

• TensorSketch
• complex random variables
• variance reduction
• input-sparsity time
• polynomial kernels