Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusion

2026-07-12

AI Summary

This work addresses the lack of a unified theoretical framework for diffusion models to handle mixed discrete-continuous data. The authors propose the Sticky Jump Diffusion (SJD) model, which couples discrete anchor states and continuous diffusion through a continuous-time Markov process. Leveraging flux balance principles, they derive a score-driven stochastic differential equation (SDE) and a reverse jump kernel. A novel denoising hazard matching mechanism is introduced to jointly estimate the score function and reverse hazard rate from a single classifier. The framework unifies existing paradigms—such as masked and continuous diffusion—as limiting cases of SJD. By designing debonding kernels as structural inductive biases, the method outperforms current hybrid diffusion approaches on CIFAR-10, Text8, and Sudoku, demonstrating its effectiveness and generality.

Abstract

We introduce Sticky Jump Diffusions (SJDs), continuous-time Markov processes on $\mathbb R^d$ whose discrete anchors are token embeddings. In forward time, anchors release their mass at a hazard rate and the released mass diffuses in the continuous ambient space; time reversal couples a score-driven SDE with a sticky jump kernel whose rate and destination are fixed by flux balance with the forward law. We estimate the score and the per-anchor reverse hazards from a single denoising classifier via Denoising Hazard Matching, the hazard analogue of denoising score matching, with simulation-free cross-entropy training. SJD recovers masked diffusion, continuous diffusion, and hybrid diffusion as limits. Its reversal explains features that each family treats as given: the mask of masked diffusion carries no evidence about the source token because the unsticking kernel of every anchor collapses to the same absorbing point; the terminal projection of continuous diffusion is required due to the absence of atoms in its forward marginal, without which flux balance yields no reverse jumps; and the update rules of hybrid diffusion (commit rate, destination, and drift) all follow from flux balance rather than from separate design. Beyond these limits, the unsticking kernel becomes a design space: a cross-position blending corrupts each position toward a blend of its neighbors' clean values or embeddings, turning dependency structure such as spatial locality or a constraint graph into an inductive bias of the corruption itself, and improves over the identity-kernel hybrid on CIFAR-10, Text8, and Sudoku.

Problem

• Sticky Jump Diffusions
• masked diffusion
• continuous diffusion
• hybrid diffusion
• denoising

Innovation

• Sticky Jump Diffusions
• Denoising Hazard Matching
• flux balance
• unsticking kernel
• hybrid diffusion