Kernel Learning for Regression via Quantum Annealing Based Spectral Sampling

2026-01-13

AI Summary

This work proposes a novel approach that directly embeds quantum annealing into kernel learning to enhance the expressiveness of translation-invariant kernels for regression tasks. Leveraging Bochner’s theorem, the spectral distribution is modeled using a restricted Boltzmann machine, and noisy, finite-temperature quantum annealers are employed for efficient sampling. Discrete samples are mapped to continuous frequencies via a Gaussian–Bernoulli transformation, yielding data-adaptive random Fourier features for Nadaraya–Watson regression. The method innovatively introduces non-negative squared kernel weights to mitigate the vanishing denominator issue in conventional random Fourier features and to improve weight contrast. Experiments demonstrate significant reductions in training loss, improved kernel matrix structure, and superior performance over Gaussian kernel baselines in terms of both R² and RMSE across multiple regression benchmarks, with inference accuracy consistently increasing as the number of features grows.

Technology Category

• Machine Learning: Kernel Methods
• Search and Optimization: Learning to Search
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

While quantum annealing (QA) has been developed for combinatorial optimization, practical QA devices operate at finite temperature and under noise, and their outputs can be regarded as stochastic samples close to a Gibbs--Boltzmann distribution. In this study, we propose a QA-in-the-loop kernel learning framework that integrates QA not merely as a substitute for Markov-chain Monte Carlo sampling but as a component that directly determines the learned kernel for regression. Based on Bochner's theorem, a shift-invariant kernel is represented as an expectation over a spectral distribution, and random Fourier features (RFF) approximate the kernel by sampling frequencies. We model the spectral distribution with a (multi-layer) restricted Boltzmann machine (RBM), generate discrete RBM samples using QA, and map them to continuous frequencies via a Gaussian--Bernoulli transformation. Using the resulting RFF, we construct a data-adaptive kernel and perform Nadaraya--Watson (NW) regression. Because the RFF approximation based on $\cos(\bm{\omega}^{\top}\Delta\bm{x})$ can yield small negative values and cancellation across neighbors, the Nadaraya--Watson denominator $\sum_j k_{ij}$ may become close to zero. We therefore employ nonnegative squared-kernel weights $w_{ij}=k(\bm{x}_i,\bm{x}_j)^2$, which also enhances the contrast of kernel weights. The kernel parameters are trained by minimizing the leave-one-out NW mean squared error, and we additionally evaluate local linear regression with the same squared-kernel weights at inference. Experiments on multiple benchmark regression datasets demonstrate a decrease in training loss, accompanied by structural changes in the kernel matrix, and show that the learned kernel tends to improve $R^2$ and RMSE over the baseline Gaussian-kernel NW. Increasing the number of random features at inference further enhances accuracy.

Problem

• quantum annealing
• kernel learning
• regression
• random Fourier features
• spectral sampling

Innovation

• Quantum Annealing
• Kernel Learning
• Random Fourier Features
• Restricted Boltzmann Machine
• Nadaraya–Watson Regression