State-of-art minibatches via novel DPP kernels: discretization, wavelets, and rough objectives

2026-05-13

AI Summary

Existing subsampling methods based on Determinantal Point Processes (DPPs) struggle to construct continuous DPPs that simultaneously achieve favorable variance reduction properties and lack efficient, structure-preserving discretization schemes. This work proposes a novel wavelet-based continuous DPP and introduces a general discretization framework that converts continuous kernels into low-rank discrete kernels while preserving their variance decay characteristics. The approach is the first to enable DPP-based subsampling for target functions with arbitrarily low regularity and provides explicit convergence rates that depend on the function’s smoothness. The proposed wavelet DPP outperforms existing methods both theoretically and empirically in terms of accuracy and efficiency, substantially broadening the applicability and effectiveness of DPPs in machine learning subsampling tasks.

Abstract

Determinantal point processes (DPPs) have emerged as a kernelized alternative to vanilla independent sampling for generating efficient minibatches, coresets and other parsimonious representations of large-scale datasets. While theoretical foundations and promising empirical performance have been demonstrated, there are two challenges for current proposals for DPP-based coresets or minibatches. The first is the need for families of DPPs with certain key variance reduction properties, usually constructed in a continuous setting, of which there are few known examples. The second is the need for an ad-hoc construction of a discrete DPP defined on a given dataset, that inherits such variance reduction. In this work, we contribute to the programme of establishing DPPs as a subsampling toolbox for ML by advancing on these two fronts. First, we propose new DPPs on the Euclidean space based on wavelets, with provably better accuracy guarantees than the best known rates. Second, we introduce a general method to convert such continuous DPPs, which are more amenable to proving analytical statements, into discrete kernels, which are pertinent for subsampling tasks such as minibatch and coreset constructions. This conversion mechanism simultaneously preserves the desired variance decay and reveals a low-rank decomposition of the discrete kernel, which makes sampling the corresponding DPP computationally inexpensive. En route, we enlarge the class of ML tasks amenable to improvements via DPP-based minibatches and coresets to include objective functions with arbitrarily low regularity, and rate guarantees that explicitly adapt to this regularity.

Problem

• Determinantal Point Processes
• Minibatch
• Coreset
• Variance Reduction
• Subsampling

Innovation

• determinantal point processes
• wavelets
• discretization
• variance reduction
• low-rank kernel