Distance Marching for Generative Modeling

2026-02-03

AI Summary

This work addresses the ambiguity in time-unconditional generative models, where the absence of temporal information leads to multiple denoising directions for the same noise input, thereby weakening the supervision signal. Inspired by distance field modeling, the authors propose Distance Marching—a method that employs a loss function emphasizing proximity to the data manifold, combined with deterministic sampling and an early-stopping mechanism to guide denoising trajectories toward more accurate convergence. As the first approach to integrate distance fields into generative modeling, it enables efficient inference without time conditioning and supports out-of-distribution (OOD) detection. The method achieves a 13.5% average FID improvement on CIFAR-10 and ImageNet, and on class-conditional ImageNet generation, it surpasses flow-matching approaches using only 60% of the sampling steps, yielding a 13.6% FID reduction.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Time-unconditional generative models learn time-independent denoising vector fields. But without time conditioning, the same noisy input may correspond to multiple noise levels and different denoising directions, which interferes with the supervision signal. Inspired by distance field modeling, we propose Distance Marching, a new time-unconditional approach with two principled inference methods. Crucially, we design losses that focus on closer targets. This yields denoising directions better directed toward the data manifold. Across architectures, Distance Marching consistently improves FID by 13.5% on CIFAR-10 and ImageNet over recent time-unconditional baselines. For class-conditional ImageNet generation, despite removing time input, Distance Marching surpasses flow matching using our losses and inference methods. It achieves lower FID than flow matching's final performance using 60% of the sampling steps and 13.6% lower FID on average across backbone sizes. Moreover, our distance prediction is also helpful for early stopping during sampling and for OOD detection. We hope distance field modeling can serve as a principled lens for generative modeling.

Problem

• time-unconditional generative models
• denoising vector fields
• noise levels
• supervision signal
• data manifold

Innovation

• Distance Marching
• time-unconditional generative modeling
• distance field
• denoising vector field
• flow matching