Kernel-Gradient Drifting Models

2026-05-11

AI Summary

Existing diffusion models rely on fixed displacement directions in Euclidean space, struggling with non-Euclidean data such as spherical or discrete sequences and lacking theoretical support for general kernel functions. This work proposes a kernel gradient drift framework that leverages the kernel itself to induce drift directions, enabling a single-step generation method applicable to arbitrary kernels and naturally extending to Riemannian manifolds and discrete domains. By uncovering a score-difference structure in the drift dynamics under general kernels, the approach achieves identifiability for characteristic kernels and unifies continuous and discrete data generation through Fisher–Rao geometry. The method attains state-of-the-art single-step performance—without distillation—on tasks involving spherical geographic data, promoter DNA sequences, and molecular generation.

Abstract

We propose kernel-gradient drifting, a one-step generative modeling framework that replaces the fixed Euclidean displacement direction in drifting models with directions induced by the kernel itself. Standard drifting is attractive because it enables fast, high-quality generation without distilling a large pretrained diffusion model, but its theory is currently understood mainly for Gaussian kernels, where the drift coincides with smoothed score matching and is identifiable. Our gradient-based reformulation exposes this score-based structure for general kernels: the resulting drift is the score difference between kernel-smoothed data and model distributions, yielding identifiability for characteristic kernels and a smoothed-KL descent interpretation of the drifting dynamics. Since kernel gradients are intrinsic tangent vectors, the same construction extends naturally to Riemannian manifolds and to discrete data via the Fisher-Rao geometry of the probability simplex. Across spherical geospatial data, promoter DNA and molecule generation, kernel-gradient drifting enables state-of-the-art one-step generation beyond the Euclidean setting without distillation.

Problem

• drifting models
• kernel gradients
• non-Euclidean generation
• score matching
• one-step generative modeling

Innovation

• kernel-gradient drifting
• score-based generative modeling
• characteristic kernels
• Riemannian manifolds
• Fisher-Rao geometry