Subsampling for supervised learning in reproducing kernel Hilbert spaces

2026-06-19

AI Summary

This study addresses the high computational cost and energy consumption of supervised learning in the era of big data by investigating nonparametric supervised learning within a reproducing kernel Hilbert space. The authors propose a Horvitz–Thompson reweighted subsampling estimator based on empirical risk minimization. Through asymptotic analysis, they derive—for the first time—the optimal subsampling probabilities under the trace norm of the covariance operator and provide an efficient plug-in implementation. Both theoretical analysis and empirical evaluations demonstrate that the proposed method substantially reduces computational overhead while preserving estimation accuracy across synthetic and real-world datasets, offering an efficient and environmentally sustainable solution for large-scale nonparametric learning.

Abstract

In the era of big data, subsampling became a common practice in statistical learning. By selecting a subgroup of individuals based on which the learner is trained, subsampling aims at reducing the computational cost and time of the estimation step, and ideally leads to a decrease of its energy consumption and carbon footprint. This work focuses on a nonparametric setting, in which the hypotheses set lies in a reproducing kernel Hilbert space, and the estimator is a minimizer of an empirical risk reweighted à la Horvitz-Thompson. By studying the asymptotic properties of this estimator, we reveal an optimal subsampling scheme (regarding the trace of the covariance operator) and show that it can be used via plug-in. A numerical study on synthetic and real-world datasets shows the practicability and the benefit of the proposed approach.

Problem

• subsampling
• supervised learning
• reproducing kernel Hilbert space
• computational efficiency
• big data

Innovation

• subsampling
• reproducing kernel Hilbert space
• Horvitz-Thompson weighting
• optimal sampling
• nonparametric learning