Second Order Drifting Models

2026-08-08

AI Summary

This work addresses the spectral rigidity inherent in first-order drift models, which stems from kernel-driven drift fields and impedes effective recovery of high-frequency details in density residuals. To overcome this limitation, the authors propose a second-order drift model that introduces an auxiliary velocity variable in phase space, thereby constructing an accelerated dynamics for density perturbations. Notably, this framework incorporates Nesterov acceleration into the drift paradigm for the first time, establishing a formal connection to convex optimization theory. The approach preserves single-step inference efficiency while alleviating spectral rigidity through phase-space modeling, Fourier-domain second-order dynamics analysis, and a semi-implicit training strategy. Empirical results demonstrate significantly faster convergence and competitive or superior performance over existing first-order baselines across distribution matching, sequence generation, and robotic control tasks.

Abstract

Drifting models are a recent class of one-step generative models that evolve the model distribution during training using a predefined sample-based drift field. Although they avoid iterative inference, their kernel-based drift fields induce frequency-dependent training dynamics: In the linearized regime, each Fourier mode of the density residual decays at a rate determined by the kernel spectrum, leading to slow recovery of fine-scale structure. We propose Second-Order Drifting Models, which lift drifting dynamics into phase space by augmenting generated samples with artificial velocity variables. We show that the resulting density perturbations obey accelerated second-order dynamics in Fourier space, connecting drifting models to the celebrated Nesterov acceleration from optimization theory. This provides a principled mechanism for mitigating the spectral stiffness of first-order drifting while preserving one-step inference. We derive a practical semi-implicit training algorithm and evaluate it on synthetic distribution matching, sequential data generation, and robotic control. Across these settings, the second-order drifting model improves convergence behavior and achieves competitive or superior performance over first-order drifting baselines.

Problem

• drifting models
• spectral stiffness
• Fourier modes
• density residual
• fine-scale structure

Innovation

• Second-Order Drifting Models
• Nesterov acceleration
• spectral stiffness
• phase space augmentation
• one-step generative models