Tensor Sketch: Fast and Scalable Polynomial Kernel Approximation

2025-05-13

AI Summary

To address the prohibitively high computational cost of polynomial kernel evaluation on large-scale, high-dimensional data, this paper proposes a randomized feature mapping method based on Tensor Sketch. Our approach is the first to integrate tensor structure with Count Sketch—a fast hashing technique—enabling efficient low-dimensional embedding via randomized projection and tensor compression. The resulting kernel approximation is computed in $O(n(d + D log D))$ time, where $n$ is the number of samples, $d$ the input dimensionality, and $D$ the target feature dimension. We provide rigorous theoretical analysis establishing tight error bounds for the approximation. Empirical evaluation demonstrates that, on datasets with millions of samples and tens of thousands of features, our method achieves kernel estimation fidelity nearly identical to exact computation, while accelerating kernel approximation by over an order of magnitude compared to baseline methods such as Random Fourier Features. This substantially enhances the scalability and practicality of nonlinear kernel methods for large-scale learning tasks.

Technology Category

• Machine Learning: Kernel Methods
• Data Mining & Knowledge Management: Data Compression
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Approximation of non-linear kernels using random feature maps has become a powerful technique for scaling kernel methods to large datasets. We propose extit{Tensor Sketch}, an efficient random feature map for approximating polynomial kernels. Given $n$ training samples in $R^d$ Tensor Sketch computes low-dimensional embeddings in $R^D$ in time $BO{n(d+D log{D})}$ making it well-suited for high-dimensional and large-scale settings. We provide theoretical guarantees on the approximation error, ensuring the fidelity of the resulting kernel function estimates. We also discuss extensions and highlight applications where Tensor Sketch serves as a central computational tool.

Problem

• Efficient approximation of polynomial kernels
• Scalable for high-dimensional large datasets
• Theoretical guarantees on approximation error

Innovation

• Random feature maps for kernel approximation
• Efficient polynomial kernel approximation
• Low-dimensional embeddings for large datasets